Pilot: Identifying key research

()

Test-case research for proof of concept

Identify a small set of papers or projects as representative first-cases; use to help test the system we are building in a concrete manner.

In doing the above, we are also collecting a longer list of key papers, projects, authors, topics, issues, etc.

Steps taken

1. Post on EA Forum (and other places) and present form (see view at bottom of this section) promoting our call for papers further, with bounty.

The most pivotal empirical pieces of research ... you would like to see red-teamed/assessed? - EA Forum

Rules for bounty HERE

2. Search for most-cited papers (within our domain) among EA-aligned researchers and organizations.

3. Dig into existing lists, reviews, and syllabi, such as:

• GPI research agenda (includes many posed questions)

• Open Philanthropy "questions that might affect our grantmaking" (needs updating? few academic links)

• The EA Behavioral Science Newsletter

• Syllabi: Pablo's list; Economics focus list; David Rhys-Bernard's syllabus (link to my commented/highlighted version)

Consider: "Mistakes" pages?

• Givewell (mainly operational mistakes)

• ACX/Scott Alexander

Not very relevant because focused on operational issues

5. Appeal directly to authors and research groups

6. Cross-promote with How to get involved

Pivot: direct focus on NBER working papers

"Direct evaluation" track

1. Pete Slattery: "Do a complete test run using a single paper and team…" Thus, we aim to identify a small set of papers (or projects), maybe 2–3, that seem like good test and example cases, and offer a bounty for projects we choose as test cases.

2. Note that much of the work identified here has already been peer-reviewed and "published." While we envision that The Unjournal may assess papers that are already published in traditional journals, these are probably not the best case for the PoC. Thus, we de-prioritize these for now.