Conventional guidelines for referee reports

How to write a good review (general conventional guidelines)

Some key points

Cite evidence and reference specific parts of the research when giving feedback.
Justify your critiques and claims in a reasoning-transparent way, rather than merely ‘"passing judgment." Avoid comments like "this does not pass the smell test".
Provide specific, actionable feedback to the author where possible.
Try to restate the authors’ arguments, clearly presenting the most reasonable interpretation of what they have written. See steelmanning.
Be collegial and encouraging, but also rigorous. Criticize and question specific parts of the research without suggesting criticism of the researchers themselves.

We're happy for you to use whichever process and structure you feel comfortable with when writing your evaluation content.

One possible structure

Core

Briefly summarize the work in context
Highlight positive aspects of the paper and its strengths and contributions, considered in the context of existing research.
Most importantly: Identify and assess the paper's most important and impactful claim(s). Are these supported by the evidence provided? Are the assumptions reasonable? Are the authors using appropriate methods?
Note major limitations and potential ways the work could be improved; where possible, reference methodological literature and discussion and work that models what you are suggesting.

Optional/desirable

Offer suggestions for increasing the impact of the work, for incorporating the work into global priorities research and impact evaluations, and for supporting and enhancing future work.
Discuss minor flaws and their potential revisions.
Desirable: formal 'claim identification and assessment'

Please don't spend time copyediting the work. If you like, you can give a few specific suggestions and then suggest that the author look to make other changes along these lines.

Remember: The Unjournal doesn’t “publish” and doesn’t “accept or reject.” So don’t give an Accept, Revise-and-Resubmit', or Reject-type recommendation. We ask for quantitative metrics, written feedback, and expert discussion of the validity of the paper's main claims, methods, and assumptions.