FAQ for authors of research that The Unjournal selected for public evaluation, and for authors considering submitting their work to The Unjournal for evaluation
You can fill out this form to submit your work, or email contact@unjournal.org with questions .
We generally seek (aka 'reviewers') with research interests in your area and with complementary expertise. You, the author, can suggest areas you want to on.
The evaluators write detailed and helpful evaluations, and submit them either "signed" or anonymously. Using our evaluation forms, they provide quantitative ratings on several dimensions, such as methods, relevance, and communication. They predict what journal tier the research will be published in, and what tier it should be published in. Here are the Guidelines for evaluators.
These evaluations and ratings are typically made public (see unjournal.pubpub.org), but you will have the right to respond before (or after) these are posted.
To consider your research we only need a link to a publicly hosted version of your work, ideally with a DOI. We will not "publish" your paper. The fact that we are handling your paper will not limit you in any way. You can submit it to any journal before, during, or after the process.
You can request a conditional embargo by emailing us at contact@unjournal.org, or via the submission form. Please explain what sort of embargo you are asking for, and why. By default, we would like Unjournal evaluations to be made public promptly. However, we may make exceptions in special circumstances, particularly for very early-career researchers whose career prospects are particularly vulnerable.
If there is a vulnerable early-career researcher on the authorship team, we may allow authors to "embargo" the publication of the evaluation until a later date. Evaluators (referees) will be informed of this. This date can be contingent, but it should not be indefinite.
For further details on this, and some examples, see 'Conditional embargos' & exceptions.
We may ask for some of the below, but these are mainly optional. We aim to make the process very light touch for authors.
A link to a non-paywalled, hosted version of your work (in ) which .
Responses to .
We may ask for a link to data and code, if possible. Note that our project is not principally about replication, and we are not insisting on this. However, sharing code, data, and materials is .
We also allow you to respond to evaluations, and we give your response its own DOI.
By submitting your research and engaging with public evaluation, you send a powerful public signal that you are confident in your work, open to constructive criticism, and motivated to seek the truth!
And there are further benefits to :
Substantive feedback will help you improve your research. is often very hard to get, especially for young scholars. It's hard to get anyone to read your paper – we can help!
Being evaluated by The Unjournal is a sign of impact. We select our research based on potential global relevance.
Ratings are markers of credibility for your work that could help your career.
The chance to publicly respond to criticism and correct misunderstandings.
Increasing the visibility of your work, which may lead to additional citations. We publicize our evaluations and the original papers on our social media feed, and occasionally in notebook and
A connection to the Open Science/Open Access and EA/Global Priorities communities. This may lead to grant opportunities, open up new ambitious projects, and attract strong PhD students to your research groups.
A reputation as an early adopter and innovator in open science.
Prizes: You may win an Impactful Research Prize (pilot) (publicity, reputation, and substantial financial prizes). These prizes are tied, in part, to your engagement with The Unjournal.
Undervalued or updated work: Your paper may have been "under-published". Perhaps there are a limited set of prestigious journals in your field. You now see ways you could improve the research. The Unjournal can help; we will also consider 'post-peer-review publication' evaluation.
Innovative formats: Journals typically require you to submit a LaTeX or MS Word file, and to use their fussy formats and styles. You may want to use tools like Quarto that integrate your code and data, allow you to present dynamic content, and enhance reproducibility. The Unjournal , and we can evaluate research in virtually any format.
There are risks and rewards to any activity, of course. Here we consider some risks you may weigh against the benefits mentioned above.
Exclusivity
Public negative feedback
, and perhaps they might enforce these more strongly if they fear competition from The Unjournal.
However, The Unjournal is not exclusive. Having your paper reviewed and evaluated in The Unjournal will not limit your options; you can still submit your work to traditional journals.
Our evaluations are public. While there has been some movement towards open review, this is still not standard. Typically when you submit your paper, reviews are private. With The Unjournal, you might get public negative evaluations.
We think this is an acceptable risk. Most academics expect that opinions will differ about a piece of work, and everyone has received negative reviews. Thus, getting public feedback — in The Unjournal or elsewhere — should not particularly harm you or your research project.
Nonetheless, we are planning some exceptions for early-career researchers (see #can-i-ask-for-evaluations-to-be-private).
Unjournal evaluations should be seen as signals of research quality. Like all such signals, they are noisy. But submitting to The Unjournal shows you are confident in your work, and not afraid of public feedback.
Within our "Direct evaluation" track, The Unjournal directly chooses papers (from prominent archives, well-established researchers, etc.) to evaluate. We don't request authors' permission here.
As you can see in our evaluation workflow, on this track, we engage with authors especially at two points:
Informing the authors that the evaluation will take place, requesting they engage, and giving them the opportunity to request a conditional embargo or specific types of feedback.
Of particular interest: are we looking at the most recent version of the paper/project, or is there a further revised version we should be considering instead?
After the evaluations have been completed, the authors are given two weeks to respond, and have their response posted along with our 'package'. (Authors can also respond after we have posted the evaluations, and we will put their response in the same 'package', with a DOI etc.)
Once we receive unsolicited work from an author or authors, we keep it in our database and have our team decide on prioritization. If your paper is prioritized for evaluation, The Unjournal will notify you.
At present, we do not have a system to automatically share the status of author submissions with authors. We hope to put one in place. You can email us for clarification and updates.
You can still submit your work to any traditional journal.
The Unjournal aims to evaluate the most recent version of a paper. We reach out to authors to ensure we have the latest version at the start of the evaluation process.
If substantial updates are made to a paper during the evaluation process, authors are encouraged to share the updated version. We then inform our evaluators and ask if they wish to revise their comments.
If the evaluators can't or don't respond, we will note this and still link the newest version.
Authors are encouraged to respond to evaluations by highlighting major revisions made to their paper, especially those relevant to the evaluators' critiques. If authors are not ready to respond to evaluations, we can post a placeholder response indicating that responses and/or a revised version of the paper are forthcoming.
Re-evaluation: If authors and evaluators are willing to engage, The Unjournal is open to re-evaluating a revised version of a paper after publishing the evaluations of the initial version.
We share evaluations with the authors and give them a chance to respond before we make the evaluations public (and again afterward, at any point). We add these to our evaluation packages on PubPub. Evaluation manager's (public) reports and our further communications incorporate the paper, the evaluations, and the authors' responses.
Authors' responses could bring several benefits...
Personally: a chance to correct misconceptions and explain their approach and planned steps. If you spot any clear errors or omissions, we give evaluators a chance to adjust their reports in response. Your response can also help others have a more accurate and positive view of the research. This includes the evaluators, as well as future journal referees and grant funders.
For research users, to get an informed balanced perspective on how to judge the work
For other researchers, to better understand the methodological issues and approaches. This can serve to start a public dialogue and discussion to help build the field and research agenda. Ultimately, we aim to facilitate a back-and-forth between authors, evaluators, and others.
Evaluations may raise substantive doubts and questions, and make some specific suggestions, and ask about (e.g.) data, context, or assumptions. There's no need to respond to every evaluator point. Only respond where you have something substantive: clarifying doubts, explaining the justification for your particular choices, and giving your thoughts on the suggestions (which will you incorporate, or not, and why?).
A well-written author response might have a clear narrative and group responses into themes.
Try to have a positive tone (no personal attacks etc.) but avoid formality, over-politeness, or flattery. Revise-and-resubmit responses at standard journals sometimes begin each paragraph with "thank you for this excellent suggestion". Feel free to skip this; we want to focus on the substance.
Examples: We've received several detailed and informative author responses, such as: