LogoLogo
  • The Unjournal
  • An Introduction to The Unjournal
    • Content overview
    • How to get involved
      • Brief version of call
      • Impactful Research Prize (pilot)
      • Jobs and paid projects with The Unjournal
        • Advisory/team roles (research, management)
        • Administration, operations and management roles
        • Research & operations-linked roles & projects
        • Standalone project: Impactful Research Scoping (temp. pause)
      • Independent evaluations (trial)
        • Reviewers from previous journal submissions
    • Organizational roles and responsibilities
      • Unjournal Field Specialists: Incentives and norms (trial)
    • Our team
      • Reinstein's story in brief
    • Plan of action
    • Explanations & outreach
      • Press releases
      • Outreach texts
      • Related articles and work
    • Updates (earlier)
      • Impactful Research Prize Winners
      • Previous updates
  • Why Unjournal?
    • Reshaping academic evaluation: Beyond accept/reject
    • Promoting open and robust science
    • Global priorities: Theory of Change (Logic Model)
      • Balancing information accessibility and hazard concerns
    • Promoting 'Dynamic Documents' and 'Living Research Projects'
      • Benefits of Dynamic Documents
      • Benefits of Living Research Projects
    • The File Drawer Effect (Article)
    • Open, reliable, and useful evaluation
      • Multiple dimensions of feedback
  • Frequently Asked Questions (FAQ)
    • For research authors
    • Evaluation ('refereeing')
    • Suggesting and prioritizing research
  • Our policies: evaluation & workflow
    • Project submission, selection and prioritization
      • What research to target?
      • What specific areas do we cover?
      • Process: prioritizing research
        • Prioritization ratings: discussion
      • Suggesting research (forms, guidance)
      • "Direct evaluation" track
      • "Applied and Policy" Track
      • 'Conditional embargos' & exceptions
      • Formats, research stage, publication status
    • Evaluation
      • For prospective evaluators
      • Guidelines for evaluators
        • Why these guidelines/metrics?
        • Proposed curating robustness replication
        • Conventional guidelines for referee reports
      • Why pay evaluators (reviewers)?
      • Protecting anonymity
    • Mapping evaluation workflow
      • Evaluation workflow – Simplified
    • Communicating results
    • Recap: submissions
  • What is global-priorities-relevant research?
  • "Pivotal questions"
    • ‘Operationalizable’ questions
    • Why "operationalizable questions"?
  • Action and progress
    • Pilot steps
      • Pilot: Building a founding committee
      • Pilot: Identifying key research
      • Pilot: Setting up platforms
      • Setting up evaluation guidelines for pilot papers
      • 'Evaluators': Identifying and engaging
    • Plan of action (cross-link)
  • Grants and proposals
    • Survival and Flourishing Fund (successful)
    • ACX/LTFF grant proposal (as submitted, successful)
      • Notes: post-grant plan and revisions
      • (Linked proposals and comments - moved for now)
    • Unsuccessful applications
      • Clearer Thinking FTX regranting (unsuccessful)
      • FTX Future Fund (for further funding; unsuccessful)
      • Sloan
  • Parallel/partner initiatives and resources
    • eLife
    • Peer Communities In
    • Sciety
    • Asterisk
    • Related: EA/global priorities seminar series
    • EA and EA Forum initiatives
      • EA forum peer reviewing (related)
      • Links to EA Forum/"EA journal"
    • Other non-journal evaluation
    • Economics survey (Charness et al.)
  • Management details [mostly moved to Coda]
    • Governance of The Unjournal
    • Status, expenses, and payments
    • Evaluation manager process
      • Choosing evaluators (considerations)
        • Avoiding COI
        • Tips and text for contacting evaluators (private)
    • UJ Team: resources, onboarding
    • Policies/issues discussion
    • Research scoping discussion spaces
    • Communication and style
  • Tech, tools and resources
    • Tech scoping
    • Hosting & platforms
      • PubPub
      • Kotahi/Sciety (phased out)
        • Kotahi: submit/eval/mgmt (may be phasing out?)
        • Sciety (host & curate evals)
    • This GitBook; editing it, etc
    • Other tech and tools
      • Cryptpad (for evaluator or other anonymity)
      • hypothes.is for collab. annotation
Powered by GitBook
On this page
  • In brief
  • Overview and call
  • Evaluators
  • Projects and papers
  • Contact us

Was this helpful?

Export as PDF
  1. An Introduction to The Unjournal

How to get involved

PreviousContent overviewNextBrief version of call

Last updated 3 months ago

Was this helpful?

In brief

(see our ) wants your involvement, help, and feedback. We offer rewards and strive to compensate people for their time and effort.

  1. Join our team: Complete to apply for our...

    1. Evaluator pool: to be eligible to be commissioned and paid to evaluate and rate research, mainly in quantitative social science and policy

    2. Field specialist teams: help identify, prioritize, and manage research evaluation in a particular field or cause area. A related lower-commitment role: help suggest, prioritize, and discuss research>

    3. Management team or advisory board, to be part of our decision-making

  2. Suggest research for us to assess using . We offer bounty rewards. Submit your own research , or by contacting

  3. Do an Independent Evaluation to build your portfolio, receive guidance, and be eligible for promotion and prizes. See details Independent evaluations (trial)

  4. Suggest "Pivotal questions" for us to focus on

Give us feedback: Is anything unclear? What could be improved? Email contact@unjournal.org. We will offer rewards for the most useful suggestions.

Overview and call

is the founder and co- of The Unjournal. The organization is currently looking for field specialists and evaluators, as well as suggestions for relevant work for The Unjournal to evaluate.

The Unjournal is building a system for credible, public, journal-independent feedback and evaluation of research.

Briefly, The Unjournal’s basic process is:
  • Identify, invite, or select contributions of relevant research that on any open platform or archive in any format.

  • Pay evaluators to give careful feedback on this work, with prizes and incentives for strong evaluation work.

  • Elicit quantifiable and comparable metrics of research quality as credible measures of value (see: ). Synthesize the results of these evaluations in useful ways.

  • Publicly post and link all reviews of the work. Award financial prizes for the work judged strongest.

  • Allow evaluators to choose if they wish to remain anonymous or to "sign" their reviews.

  • Aim to be as transparent as possible in these processes.

We maintain an open call for participants for several different roles:

  1. A pool of Evaluators (who will be paid for their time and their work; we also draw evaluators from outside this pool)

Some particular research area/field priorities (Sept. 2024)

We're interested in researchers and research-users who want to help us prioritize work for evaluation, and manage evaluations, considering

... research in any social science/economics/policy/impact-assessment area, and

... research with the potential to be among the most globally-impactful.

Some particular areas where we are hoping to expand our expertise (as of 15 Aug 2023) include:

- Biological & pandemic risk

- AI governance, AI safety

- Long-term trends, demography

- Macroeconomics/growth/(public) finance

- Quantitative political science (voting, lobbying, etc.)

- Social impact of new technology (including AI)

Evaluators

We will reach out to evaluators (a.k.a. "reviewers") on a case-by-case basis, appropriate for each paper or project being assessed. This is dependent on expertise, the researcher's interest, and a lack of conflict of interest.

Who we are looking for: We are putting together a list of people interested in being an evaluator and doing paid referee work for The Unjournal. We generally prioritize the pool of evaluators who signed up for our database before reaching out more widely.

Projects and papers

Contact us

(involving honorariums for time spent)

members (no time commitment)

(who will often also be on the Advisory Board)

(low commitment)

You can express your interest (and enter our database) .

Time commitment: Case-by-case basis. For each evaluation, for the amount of time to spend.

Compensation: We pay a minimum of $200 (updated Aug. 2024) for a prompt and complete evaluation, $400 for experienced evaluators. We offer additional prizes and incentives, and are committed to an average compensation of at least $450 per evaluator. .

Interested? Please fill out (about 3–5 min, same form for all roles or involvement).

Ready to get started doing evaluations and building a track record? See our new Independent evaluations (trial) initiative, offering prizes and recognition for the best work. You can evaluate work in our , or suggest and evaluate work.

We are looking for high-quality, globally pivotal research projects to evaluate, particularly those embodying open science practices and innovative formats. We are putting out a call for relevant research. Please suggest research . (We offer bounties and prizes for useful suggestions – .) For details of what we are looking for, and some potential examples, and accompanying links.

You can also put .

If you are interested in discussing any of the above in person, please email us () to arrange a conversation.

Note: This is under continual refinement; see our for more details.

The roles are explained in more detail here.
here
See here for more details
this form
public database
here
see this post
forward your own work
policies
this form
this form
here
David Reinstein
team
evaluator guidelines
The Unjournal
contact@unjournal.org
contact@unjournal.org
In a nutshell
#unjournal-research-affiliates-ura
Management Committee members
Advisory Board
Field Specialists
Unjournal Research Affiliates
here are some guidelines