LogoLogo
  • The Unjournal
  • An Introduction to The Unjournal
    • Content overview
    • How to get involved
      • Brief version of call
      • Impactful Research Prize (pilot)
      • Jobs and paid projects with The Unjournal
        • Advisory/team roles (research, management)
        • Administration, operations and management roles
        • Research & operations-linked roles & projects
        • Standalone project: Impactful Research Scoping (temp. pause)
      • Independent evaluations (trial)
        • Reviewers from previous journal submissions
    • Organizational roles and responsibilities
      • Unjournal Field Specialists: Incentives and norms (trial)
    • Our team
      • Reinstein's story in brief
    • Plan of action
    • Explanations & outreach
      • Press releases
      • Outreach texts
      • Related articles and work
    • Updates (earlier)
      • Impactful Research Prize Winners
      • Previous updates
  • Why Unjournal?
    • Reshaping academic evaluation: Beyond accept/reject
    • Promoting open and robust science
    • Global priorities: Theory of Change (Logic Model)
      • Balancing information accessibility and hazard concerns
    • Promoting 'Dynamic Documents' and 'Living Research Projects'
      • Benefits of Dynamic Documents
      • Benefits of Living Research Projects
    • The File Drawer Effect (Article)
    • Open, reliable, and useful evaluation
      • Multiple dimensions of feedback
  • Frequently Asked Questions (FAQ)
    • For research authors
    • Evaluation ('refereeing')
    • Suggesting and prioritizing research
  • Our policies: evaluation & workflow
    • Project submission, selection and prioritization
      • What research to target?
      • What specific areas do we cover?
      • Process: prioritizing research
        • Prioritization ratings: discussion
      • Suggesting research (forms, guidance)
      • "Direct evaluation" track
      • "Applied and Policy" Track
      • 'Conditional embargos' & exceptions
      • Formats, research stage, publication status
    • Evaluation
      • For prospective evaluators
      • Guidelines for evaluators
        • Why these guidelines/metrics?
        • Proposed curating robustness replication
        • Conventional guidelines for referee reports
      • Why pay evaluators (reviewers)?
      • Protecting anonymity
    • Mapping evaluation workflow
      • Evaluation workflow – Simplified
    • Communicating results
    • Recap: submissions
  • What is global-priorities-relevant research?
  • "Pivotal questions"
    • ‘Operationalizable’ questions
    • Why "operationalizable questions"?
  • Action and progress
    • Pilot steps
      • Pilot: Building a founding committee
      • Pilot: Identifying key research
      • Pilot: Setting up platforms
      • Setting up evaluation guidelines for pilot papers
      • 'Evaluators': Identifying and engaging
    • Plan of action (cross-link)
  • Grants and proposals
    • Survival and Flourishing Fund (successful)
    • ACX/LTFF grant proposal (as submitted, successful)
      • Notes: post-grant plan and revisions
      • (Linked proposals and comments - moved for now)
    • Unsuccessful applications
      • Clearer Thinking FTX regranting (unsuccessful)
      • FTX Future Fund (for further funding; unsuccessful)
      • Sloan
  • Parallel/partner initiatives and resources
    • eLife
    • Peer Communities In
    • Sciety
    • Asterisk
    • Related: EA/global priorities seminar series
    • EA and EA Forum initiatives
      • EA forum peer reviewing (related)
      • Links to EA Forum/"EA journal"
    • Other non-journal evaluation
    • Economics survey (Charness et al.)
  • Management details [mostly moved to Coda]
    • Governance of The Unjournal
    • Status, expenses, and payments
    • Evaluation manager process
      • Choosing evaluators (considerations)
        • Avoiding COI
        • Tips and text for contacting evaluators (private)
    • UJ Team: resources, onboarding
    • Policies/issues discussion
    • Research scoping discussion spaces
    • Communication and style
  • Tech, tools and resources
    • Tech scoping
    • Hosting & platforms
      • PubPub
      • Kotahi/Sciety (phased out)
        • Kotahi: submit/eval/mgmt (may be phasing out?)
        • Sciety (host & curate evals)
    • This GitBook; editing it, etc
    • Other tech and tools
      • Cryptpad (for evaluator or other anonymity)
      • hypothes.is for collab. annotation
Powered by GitBook
On this page
  • Management committee members
  • Advisory board (AB) members
  • Field Specialists and Unjournal Research Affiliates
  • What Field Specialists (FS's) and Unjournal Research Affiliates (URAs) do, specifically
  • Benefits of being a Field Specialist or URA
  • Field Specialists: specifics and norms
  • Unjournal Research Affiliates: specifics and norms
  • Contact us

Was this helpful?

Export as PDF
  1. An Introduction to The Unjournal

Organizational roles and responsibilities

PreviousReviewers from previous journal submissionsNextOur team

Last updated 4 months ago

Was this helpful?

14 Jan 2025: The Unjournal is still looking to build our team and evaluator pool. Please consider the roles below and or contact us at contact@unjournal.org.

Management committee members

Activities of those on the management committee may involve a combination of the following (although you can choose your focus):

  • Contributing to the decision-making process regarding research focus, reviewer assignment, and prize distribution.

  • Collaborating with other committee members on the establishment of rules and guidelines, such as determining the metrics for research evaluation and defining the mode of assessment publication.

  • Helping plan The Unjournal’s future path.

  • Helping monitor and prioritize research for The Unjournal to evaluate (i.e., acting as a ; see further discussion below). Acting as an for research in your area.

Time commitment: A minimum of 15–20 hours per year.

Compensation: We have funding for a $57.50 per hour honorarium for the first 20 hours, with possible compensation . will be further compensated (at roughly $300–$450 per paper).

Who we are looking for: All applicants are welcome. We are especially interested in those involved in global priorities research (and related fields), policy research and practice, open science and meta-science, bibliometrics and scholarly publishing, and any other academic research. We want individuals with a solid interest in The Unjournal project and its goals, and the ability to meet the minimal time commitment. Applying is extremely quick, and those not chosen will be considered for other roles and work going forward.

Advisory board (AB) members

Beyond direct roles within The Unjournal, we're building a larger, more passive advisory board to be part of our network, to offer occasional feedback and guidance. There is essentially no minimum time commitment for advisory board members—only opportunities to engage. We sketch some of the expectations in the fold below.

Advisory board members: expectations

As an AB member...

  • you agree to be listed on our page as being on the advisory board.

  • you have the option (but not the expectation or requirement) to join our Slack, and to check in once in a while.

  • you will be looped in for your input on some decisions surrounding The Unjournal's policies and direction. Such communications might occur once per month, and you are not obligated to respond.

  • you may be invited to occasional video meetings (again optional).

  • you are “in our system” and we may consult you for other work.

  • you will be compensated for anything that requires a substantial amount of your time that does not overlap with your regular work.

Field Specialists and Unjournal Research Affiliates

Goals: Support open evaluation and impactful, credible research, helping The Unjournal keep on top of the latest, most relevant, influential, and globally-consequential work. Help raise awareness of The Unjournal, fostering a transition away from traditional journal peer review towards a more transparent, efficient, and rigorous approach.

What Field Specialists (FS's) and Unjournal Research Affiliates (URAs) do, specifically

Source, prioritize, and suggest research

Focus on a particular area of research, policy, or impactful outcome, in collaboration with other members of our area-focused teams.

Keep track of new or under-considered research with potential for impact and explain and assess the extent to which The Unjournal can add value by commissioning its evaluation. Provide a brief explanation of why each paper matters and how it aligns with The Unjournal’s approach.

Scope: Perhaps three papers per academic term; FS are encouraged to suggest more. This comes with compensation and incentives

Prioritize research submitted by authors or suggested by other team members

Write and share "second opinion" assessments of this research, considering the importance and relevance for our evaluation. Vote on a short list of papers in your area to determine whether we should commission these for evaluation. Attend (very occassional) 'field group meetings' to discuss this.

Evaluation management (FS)

Field Specialists may serve as 'evaluation managers' (with additional compensation). Evaluation managers help us identify and commission evaluators for work we have prioritized, help manage this evaluation process, and write a summary report on the evaluations

Represent The Unjournal

Understand our mission and be ready to explain it to colleagues and (optionally) on social media.

Field specialist and URA "area teams"

We are organizing several teams of field specialists (and management and advisory board members). These teams will hold occasional online meetings (perhaps every 3 months) to discuss research to prioritize, and to help coordinate 'who covers what'. If team members are interested, further discussions, meetings, and seminars might be arranged, but this is very much optional.

As of Dec 2024, we have the following teams (organized around fields and outcomes)

  1. Development economics and global health and development

  2. Economics, welfare, and governance

  3. Psychology, behavioral science, and attitudes

  4. Innovation and meta-science, impact of emerging technologies, catastrophic risks

  5. Animal welfare: markets, attitudes

  6. Environmental economics

Other teams are being organized or considered

If you become a field specialist or URA, what happens next?

You will be asked to fill out to let us know what fields, topics, and sources of research you would like to "monitor" or dig into to help identify and curate work relevant for Unjournal evaluation, as well as outlining your areas of expertise (the form takes perhaps 5–20 minutes).

This survey helps us understand when to contact you to ask if you want to be an evaluation manager on a paper we have prioritized for evaluation.

Guided by this survey form (along with discussions we will have with you, and coordination with the team), we will develop an “assignment” that specifies the area you will cover. We will try to divide the space and not overlap between field specialists. This scope can be as broad or focused as you like.

Within your area, you keep a record of the research that seems relevant (and why, and what particularly needs evaluation, etc.) and enter it in our database. (Alternatively, you can pass your notes to us for recording.)

"Monitoring" a research area or source as a field specialist or URA

The Unjournal's field specialists choose an area they want to monitor. By this we mean that a field specialist will

  • Keep an eye on designated sources (e.g., particular working paper series) and fields (or outcomes or area codes), perhaps every month or so; consider new work, dig into archives

  • Let us know what you have been able to cover; if you need to reduce the scope, we can adjust it

  • Suggest/Input work into our database … papers/projects/research that seems relevant for The Unjournal to evaluate. Give some quick ‘prioritization ratings’

  • If you have time, give a brief on why this work relevant for UJ (impactful, credible, timely, open presentation, policy-relevant, etc) and what areas need particular evaluation and feedback

Benefits of being a Field Specialist or URA

  • Public Leadership: It publicly signals your commitment to open science principles, innovation, and research impact.

  • Professional Credibility & Recognition: URAs are chosen through a selective process, and will be publicly acknowledged. URAs gain valuable experience, strengthen their CV, and position themselves for potential future opportunities with The Unjournal (e.g., Field Specialist or management roles).

  • Networking & Community: Build contacts with others in your field and with related interests. Stay on top of cutting-edge research, discuss and understand its value and connection to the field and to policy/impact.

  • Influencing the Agenda: URAs help shape what research The Unjournal evaluates and promotes, furthering our mission of global impact, and advocating for your own priorities.

Field Specialists: specifics and norms

Who we are looking for: For the FS roles, we are seeking active researchers, practitioners, and stakeholders with a strong publication record and/or involvement in the research and/or research-linked policy and prioritization processes. For the AB, also people with connections to academic, governmental, or relevant non-profit institutions, and/or involvement in open science, publication, and research evaluation processes. People who can offer relevant advice, experience, guidance, or help communicate our goals, processes, and progress.

Unjournal Research Affiliates: specifics and norms

Application criteria: Applicants will selected based on (1) their ability to assess, discuss, and communicate the global impact of research, (2) their understanding of The Unjournal approach, (3) their research skills, interests, and experience.

Low Time Commitment: This role is designed to fit into a busy academic or research schedule. You’ll stay involved at a comfortable level without overextending your commitments.

Expectations for URAs

  1. Recommend 2–3 papers each academic term that you think have the potential to make a substantial impact. Provide a brief explanation of why each paper matters and how it aligns with The Unjournal’s approach. Earn compensation for your recommendations.

  2. Provide Second Opinions & Prioritization: Complete three “second opinion” assessments each term, offering a concise evaluation of a paper’s relevance and importance. Contribute to broader discussions about research prioritization.

  3. Vote & Engage: Participate in monthly voting on a short list of papers. Stay semi-active on Slack and Coda (with no strict requirement to check in regularly, beyond voting).

  4. Represent The Unjournal: Understand our mission and be ready to explain it to colleagues and on social media. Attend at least one Unjournal meeting (one hour) per year to stay connected.

  5. Optional: Consult on evaluation management Occasionally advise on evaluator suggestions for papers in your field (additional compensation may apply).

Why are we offering the URA role?

Why are we offering this role? Early career researchers and PhD students may be interested in getting involved but worried about making . But getting busy students and researchers involved even minimally could keep us in touch with the cutting edge of research and help us forge collaborations in academia. So we're offering the role, which is similar to the Field Specialist role, but with less responsibility and time commitment.

Contact us

Help track and curate this list of prioritized research, considering and discussing specific areas and issues that merit evaluation. We maintain a public database of research of potential interest .

We will compensate you for the time you spend on this process (details tbd), particularly to the extent that the time you spend does not contribute to your other work or research. (See .)

Compensation: Incentive pay for recommending and evaluating papers. Be compensated ($) for research surveying and scoping you may are already be doing. See compensation details .

See the page.

Time commitment: There is no specific time obligation—only opportunities to engage. We may also consult you occasionally on your areas of expertise. Perhaps 1–4 hours a month is a reasonable starting expectation for people already involved in doing or using research, plus potential additional paid assignments. Our document also provides some guidance on the nature of work and the time involved.

Compensation: We aim to fairly compensate people for time spent on work done to support The Unjournal, and to provide incentives for suggesting and helping to prioritize research for evaluation. See the page. will be compensated at roughly $300–$450 per project.

Interested? (about 3–5 min, using the same form for all roles).

Compensation This role comes with , but URAs are eligible for incentive compensation for suggesting papers, for doing high-value assessment for prioritization, and more (see ).

for this and other roles.

Self-contained public release:

If you are interested in discussing any of the above in person, please email us () to arrange a conversation.

We invite you to (the same as that linked above) to leave your contact information and outline which parts of the project interest you.

Note: These descriptions are under continual refinement; see our for more details.

here
incentives and norms
here
here
Norms and Compensation
Incentives and norms
Norms and Compensation
Evaluation management work
Please fill out this form
here
Apply here
Open Call: Unjournal Research Affiliates (URA)
fill in this form
policies
express your interest here
evaluation manager
Evaluation management work
field specialist
contact@unjournal.org