# For prospective evaluators ***Thanks for your interest in evaluating research for***** The Unjournal!** ## **Who we are** *The Unjournal* is a nonprofit organization started in mid-2022. We commission experts to publicly evaluate and rate research. Read more about us [here](/the-unjournal-project-and-communication-space/~/changes/531/readme-1.md). ## What we are asking you to do 1. **Write an evaluation of a specific research** [***paper or project***](#user-content-fn-1)[^1]**:** essentially a standard, high-quality referee report. 2. [**Give quantitative ratings and predictions about the** ](#user-content-fn-2)[^2]**research** by filling in a structured form. 3. Answer a short questionnaire about your background and our processes. See [Guidelines for Evaluators](/the-unjournal-project-and-communication-space/~/changes/531/policies-projects-evaluation-workflow/evaluation/guidelines-for-evaluators.md) for further details and guidance. ## Why be an evaluator? Why use your valuable time writing an *Unjournal* evaluation? There are several reasons: helping high-impact research users, supporting open science and open access, and getting recognition and financial compensation. ### Helping research users, helping science *The Unjournal's* goal is to make impactful research more rigorous, and rigorous research more impactful, while supporting open access and open science. We encourage better research by making it easier for researchers to get feedback and credible ratings. We evaluate research in high-impact areas that make a difference to global welfare. Your evaluation will: 1. **Help authors** improve their research, by giving early, high-quality feedback. 2. **Help improve science** by providing **open-access, prompt, structured, public evaluations** of impactful research. 3. **Inform funding bodies and meta-scientists** as we build a database of research quality, strengths and weaknesses in different dimensions. **Help research users** learn what research to trust, when, and how. For more on our scientific mission, see [here](/the-unjournal-project-and-communication-space/~/changes/531/benefits-and-features.md). ### Public recognition Your evaluation will be made public and given a DOI. You have the option to be identified as the author of this evaluation or to remain anonymous, as you prefer. ### Financial compensation [You will be given a $200-$400 honorarium](#user-content-fn-3)[^3] for providing a prompt[^4] and complete evaluation and feedback ($100-$300 base + $100 'promptness bonus') in line with our [expected standards](#what-we-are-asking-you-to-do). {% hint style="info" %} **Note, Aug. 2024**: we're adjusting the base compensation to reward strong work and experience. [*Minimum base compensation:* ](#user-content-fn-5)[^5] * $100 + $100 for first-time evaluators * $300 + $100 for return Unjournal evaluators and those with previous strong public review experience.\ \ We will be integrating other incentives and prizes into this, and are committed to [$450](https://x.com/450Movement) in average compensation per evaluation, including prizes. {% endhint %} You will also be eligible for monetary prizes for "most useful and informative evaluation," plus other bonuses. We currently (Feb. 2024) set aside an additional $150 per evaluation for incentives, bonuses, and prizes. See also ["submitting claims and expenses"](/the-unjournal-project-and-communication-space/~/changes/531/management-tech-details-discussion/fiscal-hosting-and-expenses.md#submitting-and-paying-expenses-claims)
Additional rewards and incentives We may occasionally offer additional payments for specifically requested evaluation tasks, or raise the base payments for particularly hard-to-source expertise. July 2023: The above is our current policy; we are working to build an effective, fair, transparent, and straightforward system of honorariums, incentives, and awards for evaluators. Feb. 204: Note that we currently set aside an additional $150 per evaluation (i.e., per evaluator) for evaluator incentives, bonuses, and prizes. This may be revised upwards or downwards in future (and this will be announced and noted).
## What do I do next? * If you have been invited to be an evaluator and want to proceed, simply respond to the email invitation that we have sent you. You will then be sent a link to our evaluation form. * To sign up for our evaluator pool, see ['how to get involved'](/the-unjournal-project-and-communication-space/~/changes/531/readme-1/call-for-participants-research.md) To learn more about our evaluation process, see[Guidelines for evaluators](/the-unjournal-project-and-communication-space/~/changes/531/policies-projects-evaluation-workflow/evaluation/guidelines-for-evaluators.md). If you are doing an evaluation, we highly recommend you read these guidelines carefully
Note on the evaluation platform (13 Feb 2024) 12 Feb 2024: We are moving to a hosted form/interface in PubPub. That form is still somewhat a work-in-progress, and may need some further guidance; we try to provide this below, but please contact us with any questions. [If you prefer](#user-content-fn-6)[^6], you can also submit your response in a Google Doc**,** and share it back with us. Click [here](https://docs.google.com/document/d/1erOQ8qiWmgAmd9WdMLmuBGoxFkUJeQo2c8pc5wFAQbk/copy) to make a new copy of that directly.
[^1]: We refer to a 'paper or project' here. We encourage the submission of a range of formats, including dynamic documents. See [Promoting 'Dynamic Documents' and 'Living Research Projects'](/the-unjournal-project-and-communication-space/~/changes/531/benefits-and-features/dynamic-documents-vs-living-projects.md). However, we will refer to this as a 'paper' throughout the discussion below, for simplicity. [^2]: You may want to glance at these tables before writing your report, to gain a sense of our priorities. [^3]: Updated Aug. 2024 [^4]: We will agree on a scheduled deadline. Generally, we aim for a three-week turnaround. Evaluations submitted after the agreed deadline (but still in a reasonable window) will earn a $300 honorarium. [^5]: In some cases we may offer more than this. We'll let evaluators know their rate in advance. [^6]: We are replacing the Google Docs with the tailored PubPub interface; the PubPub form should be more efficient and leave less room for misunderstanding. But we will still be flexible to let you link or upload your descriptive, free-form evaluation in whatever form you prefer. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://globalimpact.gitbook.io/the-unjournal-project-and-communication-space/~/changes/531/policies-projects-evaluation-workflow/evaluation/for-prospective-evaluators.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.