LogoLogo
  • The Unjournal
  • An Introduction to The Unjournal
    • Content overview
    • How to get involved
      • Brief version of call
      • Impactful Research Prize (pilot)
      • Jobs and paid projects with The Unjournal
        • Advisory/team roles (research, management)
        • Administration, operations and management roles
        • Research & operations-linked roles & projects
        • Standalone project: Impactful Research Scoping (temp. pause)
      • Independent evaluations (trial)
        • Reviewers from previous journal submissions
    • Organizational roles and responsibilities
      • Unjournal Field Specialists: Incentives and norms (trial)
    • Our team
      • Reinstein's story in brief
    • Plan of action
    • Explanations & outreach
      • Press releases
      • Outreach texts
      • Related articles and work
    • Updates (earlier)
      • Impactful Research Prize Winners
      • Previous updates
  • Why Unjournal?
    • Reshaping academic evaluation: Beyond accept/reject
    • Promoting open and robust science
    • Global priorities: Theory of Change (Logic Model)
      • Balancing information accessibility and hazard concerns
    • Promoting 'Dynamic Documents' and 'Living Research Projects'
      • Benefits of Dynamic Documents
      • Benefits of Living Research Projects
    • The File Drawer Effect (Article)
    • Open, reliable, and useful evaluation
      • Multiple dimensions of feedback
  • Frequently Asked Questions (FAQ)
    • For research authors
    • Evaluation ('refereeing')
    • Suggesting and prioritizing research
  • Our policies: evaluation & workflow
    • Project submission, selection and prioritization
      • What research to target?
      • What specific areas do we cover?
      • Process: prioritizing research
        • Prioritization ratings: discussion
      • Suggesting research (forms, guidance)
      • "Direct evaluation" track
      • "Applied and Policy" Track
      • 'Conditional embargos' & exceptions
      • Formats, research stage, publication status
    • Evaluation
      • For prospective evaluators
      • Guidelines for evaluators
        • Why these guidelines/metrics?
        • Proposed curating robustness replication
        • Conventional guidelines for referee reports
      • Why pay evaluators (reviewers)?
      • Protecting anonymity
    • Mapping evaluation workflow
      • Evaluation workflow – Simplified
    • Communicating results
    • Recap: submissions
  • What is global-priorities-relevant research?
  • "Pivotal questions"
    • ‘Operationalizable’ questions
    • Why "operationalizable questions"?
  • Action and progress
    • Pilot steps
      • Pilot: Building a founding committee
      • Pilot: Identifying key research
      • Pilot: Setting up platforms
      • Setting up evaluation guidelines for pilot papers
      • 'Evaluators': Identifying and engaging
    • Plan of action (cross-link)
  • Grants and proposals
    • Survival and Flourishing Fund (successful)
    • ACX/LTFF grant proposal (as submitted, successful)
      • Notes: post-grant plan and revisions
      • (Linked proposals and comments - moved for now)
    • Unsuccessful applications
      • Clearer Thinking FTX regranting (unsuccessful)
      • FTX Future Fund (for further funding; unsuccessful)
      • Sloan
  • Parallel/partner initiatives and resources
    • eLife
    • Peer Communities In
    • Sciety
    • Asterisk
    • Related: EA/global priorities seminar series
    • EA and EA Forum initiatives
      • EA forum peer reviewing (related)
      • Links to EA Forum/"EA journal"
    • Other non-journal evaluation
    • Economics survey (Charness et al.)
  • Management details [mostly moved to Coda]
    • Governance of The Unjournal
    • Status, expenses, and payments
    • Evaluation manager process
      • Choosing evaluators (considerations)
        • Avoiding COI
        • Tips and text for contacting evaluators (private)
    • UJ Team: resources, onboarding
    • Policies/issues discussion
    • Research scoping discussion spaces
    • Communication and style
  • Tech, tools and resources
    • Tech scoping
    • Hosting & platforms
      • PubPub
      • Kotahi/Sciety (phased out)
        • Kotahi: submit/eval/mgmt (may be phasing out?)
        • Sciety (host & curate evals)
    • This GitBook; editing it, etc
    • Other tech and tools
      • Cryptpad (for evaluator or other anonymity)
      • hypothes.is for collab. annotation
Powered by GitBook
On this page
  • EA Forum suggestions and formats
  • Peer review *on* the EA Forum?
  • Peter Slattery

Was this helpful?

Export as PDF
  1. Parallel/partner initiatives and resources
  2. EA and EA Forum initiatives

Links to EA Forum/"EA journal"

This initiative and EA/gp Unjournal will interact with the EA forum and build on initiatives coming there.

PreviousEA forum peer reviewing (related)NextOther non-journal evaluation

Last updated 1 year ago

Was this helpful?

Some of these links come from a conversation with Aaron Gertler

EA Forum suggestions and formats

  • Here's where to .

  • Here's an example of a that led us to develop a new feature.

Note: Reinstein and Hamish Huggard have worked on tools to help transform R-markdown and bookdown files. Some work can be found on (but may need some explanation).

Peer review *on* the EA Forum?

Jaime Sevilla has thoughts on creating a peer-review system for the Forum. (See embedded doc below, link .)

Peter Slattery

  1. To create a quick and easy prototype to test, you fork the EA Forum and use that fork as a platform for the Unjournal project (maybe called something like "The Journal of Social Impact Improvement and Assessment").

  2. People (ideally many from EA) would use the Forum-like interface to submit papers to this Unjournal.

  3. These papers would look like EA Forum posts, but with an included OSF link to a PDF version. Any content (e.g., slides or video) could be embedded in the submission.

  4. All submissions would be reviewed by a single admin (you?) for basic quality standards.

  5. Most drafts would be accepted to The Unjournal.

  6. Any accepted drafts would be publicly "peer reviewed." They would achieve peer-reviewed status when >x (3?) people from a predetermined or elected board of editors or experts had publicly or anonymously reviewed the paper by commenting publicly on the post. Reviews might also involve ratings the draft on relevant criteria (INT?). Public comment/review/rating would also be possible.

  7. Draft revisions would be optional but could be requested. These would simply be new posts with version X/v X appended to the title.

  8. All good comments or posts to the journal would receive upvotes, etc., so authors, editors and commentators would gain recognition, status and "points" from participation. This is sufficient for generating participation in most forums and notably lacking in most academic settings.

  9. Good papers submitted to the journal would be distinguished by being more widely read, engaged with, and praised than others. If viable, they would also win prizes. As an example, there might be a call for papers on solving issue x with a reward pool of grant/unconditional funding of up to $x for winning submissions. The top x papers submitted to The Unjournal in response to that call would get grant funding for further research.

  10. A change in rewards/incentives (from "I had a paper accepted/cited" to "I won a prize") seems to have various benefits.

  11. It still works for traditional academic metrics—grant money is arguably even more prized than citations and publication in many settings

  12. It works for non-academics who don't care about citations or prestigious journal publications.

  13. As a metric, "funds received" would probably better track researchers' actual impact than their citations and acceptance in a top journal. People won't pay for more research that they don't value, but they will cite or accept that to a journal for other reasons.

  14. Academics could of course still cite the DOIs and get citations tracked this way.

  15. Reviewers could be paid per-review by research commissioners.

  16. Here is a quick example of how it could work for the first run: Open Philanthropy calls for research on something they want to know about (e.g., interventions to reduce wild animal suffering). They commit to provide up $100,000 in research funding for good submissions and $10,000 for review support. Ten relevant experts apply and are elected to the expert editorial boards to review submissions. They will receive 300 USD per review and are expected to review at least x papers. People submit papers; these are reviewed; OP awards follow-up prizes to the winning papers. The cycle repeats with different funders, and so on.

I suppose I like the above because it seems pretty easy and actionable to do over as a test run for something to refine and scale. I estimate that I could probably do it myself if I had 6–12 months to focus on it. However, I imagine that I am missing a few key considerations as I am usually over-optimistic! Feel free to point those out and offer feedback.

suggest new Forum features
PR FAQ post
this Repo
here