# An Introduction to The Unjournal

## In a nutshell

_The Unjournal_ seeks to make rigorous research more impactful and impactful research more rigorous. We are a [team of researchers](https://www.unjournal.org/team), practitioners, and open science advocates led by [David Reinstein](https://www.davidreinstein.org/).&#x20;

_The Unjournal_ encourages better research by making it easier for researchers to get feedback and credible ratings. We coordinate and fund public journal-independent expert evaluation of hosted [_**papers and dynamically presented projects**_](#user-content-fn-1)[^1]. We publish evaluations, ratings, manager summaries, author responses, and links to evaluated research on [our PubPub page](https://unjournal.pubpub.org/).&#x20;

As the name suggests, we are not a journal!&#x20;

We work independently of traditional academic journals. We're building an open platform and a sustainable system for feedback, ratings, and assessment. We're currently focusing on quantitative work that [informs global priorities](#user-content-fn-2)[^2] in [economics, social science, and policy](#user-content-fn-3)[^3].



**How to get involved?**

We're looking for _research_ to evaluate, as well as evaluators. You can submit [your own](#user-content-fn-4)[^4] research [here](https://bit.ly/UJsubmit), or suggest research [using this form](https://coda.io/form/Suggesting-research_ddYqto0PuD0). We offer financial prizes for suggesting research we end up evaluating. If you want to be an evaluator, apply [here](https://coda.io/form/Join-the-Unjournal_dc3NLlpa-eq). You can use [the same form](https://coda.io/form/Join-the-Unjournal_dc3NLlpa-eq) to express your interest in joining our management team, advisory board, or reviewer pool. For more information, see our [how to get involved](https://globalimpact.gitbook.io/the-unjournal-project-and-communication-space/readme-1/call-for-participants-research) guide. &#x20;

\
\
**Why&#x20;**_**The Unjournal**_**?**\
\
Peer review is great, but conventional academic publication processes are wasteful, slow, and rent-extracting. They discourage innovation and prompt researchers to focus more on "gaming the system" than on the quality of their research. We will provide an immediate alternative, and at the same time, offer a bridge to a more efficient, informative, useful, and transparent research evaluation system.



**Does&#x20;**_**The Unjournal**_**&#x20;charge any fees?**

No. We're a US-registered tax-exempt 501(c)(3) nonprofit, and we don't charge fees for anything. We compensate evaluators for their time and we even award prizes for strong research and evaluation work, in contrast to most traditional journals. We do so thanks to funding from the [Effective Altruism Infrastructure Fund](https://funds.effectivealtruism.org/funds/ea-community), [Long-Term Future Fund](https://funds.effectivealtruism.org/funds/far-future) and [Survival and Flourishing Fund](https://survivalandflourishing.fund/).&#x20;

At some point in the future, we might consider sliding-scale fees for people or organizations submitting their work for _Unjournal_ evaluation, or for other services. If we do this, it would simply be a way to cover the compensation we pay evaluators and to cover our actual costs. Again, we are a _nonprofit_ and we will stay that way.

## How do we do this?

1. **Research submission/identification and selection:** We identify, solicit, and select relevant research work to be hosted on any open platform in any format [that can gain a time-stamped DOI.](#user-content-fn-5)[^5] Authors are encouraged to present their work in the ways they find most comprehensive and understandable. We support the use of [dynamic documents](https://berkeley-scf.github.io/tutorial-dynamic-docs/) and other formats that foster replicability and open science. (See: [the benefits of dynamic docs](benefits-and-features/dynamic-documents-vs-living-projects/benefits-of-dynamic-documents)).&#x20;
2. **Paid evaluators (AKA "reviewers"):** We compensate evaluators (essentially, reviewers) for providing thorough feedback on this work. (Read more: [Why do we pay?](policies-projects-evaluation-workflow/evaluation/why-pay-evaluators-reviewers))
3. **Eliciting quantifiable and comparable metrics:** We aim to establish and generate credible measures of research quality and usefulness. We benchmark these against traditional previous measures (such as journal tiers) and assess the reliability, consistency, and predictive power of these measures. (Read more: [Why quantitative metrics?](../policies-projects-evaluation-workflow/evaluation/guidelines-for-evaluators/why-these-guidelines#why-numerical-ratings))
4. **Public evaluation:** We publish the evaluation packages (including reports, ratings,  author responses, and manager summaries) on our [PubPub community](http://unjournal.pubpub.org).  Making evaluation public facilitates dialogue, and supports transparency[^6], impact[^7], understanding[^8], and [research progress](#user-content-fn-9)[^9].
5. **Linking, not publishing:** Our process is not "exclusive." Authors can submit their work to a journal (or other evaluation service) at any time. This approach also allows us to [benchmark our evaluations](#user-content-fn-10)[^10] against traditional publication outcomes.
6. **Prizes:** We award financial prizes and hold public events to recognize the most credible, impactful, useful, and insightful research, as well as strong engagement with our evaluation process.&#x20;
7. **Transparency:** We aim for maximum transparency in our processes and judgments.

<details>

<summary>This is not an original idea, and there are others in this space, but...</summary>

For example, this is closely related to ELife's ["Publish, Review, Curate" model](https://elifesciences.org/articles/64910); see their updated (Oct 2022) model [here](https://elifesciences.org/inside-elife/54d63486/elife-s-new-model-changing-the-way-you-share-your-research).  COS is also building a "[lifecycle journal](https://www.cos.io/lifecyclejournal)".  [PREReview](https://prereview.org/) promotes public journal-independent evaluation.  However, we cover a different research focus and make some different choices, discussed below. \
\
We also discuss other [parallel-partner-initiatives-and-resources](parallel-partner-initiatives-and-resources "mention"), many of whom we are building partnerships with. However, we think we are the only group funded to do this in this particular research area/focus. We are also taking a different approach to previous efforts, including funding evaluation (see [why-pay-evaluators-reviewers](policies-projects-evaluation-workflow/evaluation/why-pay-evaluators-reviewers "mention")) and asking for quantified ratings and predictions (see [guidelines-for-evaluators](policies-projects-evaluation-workflow/evaluation/guidelines-for-evaluators "mention")).

</details>

<details>

<summary><strong>Funding</strong></summary>

29 Oct 2024: We have about a 9-12 month runway, which could be extended to cover our basic activities for a longer period. We are actively applying for grants and funding.&#x20;

Our current support comes from:

[survival-and-flourishing-fund-successful](grants-and-proposals/survival-and-flourishing-fund-successful "mention"); funds deposited Summer 2023.\
\
[acx-ltff-grant-proposal-as-submitted-successfull](grants-and-proposals/acx-ltff-grant-proposal-as-submitted-successfull "mention") grant (ACX passed it to the Long Term Future Fund, who awarded it). Extended through mid-2023.\
\
We have submitted some other grant applications; e.g., see our unsuccessful [_FTX application here_](grants-and-proposals/unsuccessful-applications/ftx-future-fund-for-further-funding-unsuccessful)_;_ other grant applications are linked below. We are sharing these in the spirit of transparency.

</details>

## Change is Hard: Overcoming Academic Inertia

Academics and funders have complained about this stuff for years [and continue to do so every day on social media](https://docs.google.com/presentation/d/194u2NNvFSvc3IOfQwIrF5d4W3eFyW9GXrw_igWQOS3g/edit#slide=id.g15b6b3080d0_0_528) .  We are fairly confident our critiques of the traditional review and publication process will resonate with most readers.

So why haven't academia and the research community been able to move to something new? There is a difficult _collective action problem._ Individual researchers and universities find it risky to move unilaterally. But we believe we have a good chance of finally changing this model and moving to a better equilibrium.  We will:

* **Take risks**_:_ Many members of _The Unjournal_ management are not traditional academics; we can stick our necks out. We are also recruiting established _senior_ academics who are less professionally vulnerable.
* **Bring in new interests, external funding, and incentives**_:_ There are a range of well-funded and powerful organizations—such as the [Sloan Foundation](https://sloan.org/) and [Open Philanthropy](https://www.openphilanthropy.org/)—with a strong inherent interest in high-impact research being reliable, robust, and [reasoning-transparent](https://www.openphilanthropy.org/research/reasoning-transparency/). This support can fundamentally shift existing incentive structures.
* **Allow less risky "bridging steps"**: As noted above, _The Unjournal_ allows researchers to submit their work to traditional journals. In fact, this will provide a _benchmark_ to help build our quantitative ratings and demonstrate their value.
* **Communicate with researchers and stakeholders** to make our processes easy, clear, and useful to them.
* **Make our output useful, in the meantime**_:_ It may take years for university departments and grant funders to incorporate journal-independent evaluations as part of their metrics and reward systems. _The Unjournal_ can be somewhat patient: our evaluation, rating, feedback, and communication are already providing a valuable service to authors, policymakers, and other researchers.
* **Leverage new technology**_:_ A new set of open-access and AI-powered tools makes what we are trying to do easier, and makes[ formats other than static PDFs](#user-content-fn-11)[^11] more useful every day.
* **Reward early adopters with prizes and recognition**: We can replace "fear of standing out" with "fear of missing out." In particular, authors and research institutions that commit to publicly engaging with evaluations and critiques of their work should be commended and rewarded. And we are doing this.

## **Our objectives**

This GitBook is a knowledge base that supplements our main public page, [unjournal.org](https://unjournal.org). It serves as a platform to organize our ideas and resources and track our progress towards _our_ dual objectives:

1. Making "peer evaluation and rating" of open projects into a standard high-status outcome in academia and research, specifically within economics and social sciences. This stands in contrast to the conventional binary choice of accepting or rejecting papers to be published as PDFs and other static formats.
2. Building a cohesive and efficient system for publishing, accruing credibility, and eliciting feedback for research aligned with effective altruism and global priorities. **Our ultimate aim is to&#x20;**_**make rigorous research more impactful, and impactful research more rigorous**_**.**

## Where do I go next?

See [contents-page](readme-1/contents-page "mention")

You can also search and query this Gitbook (press control-K or command -k)



[^1]: &#x20;We encourage the submission of a range of formats, including dynamic documents. See [dynamic-documents-vs-living-projects](benefits-and-features/dynamic-documents-vs-living-projects "mention"). However, we mainly refer to these as "papers" throughout our web page, for simplicity&#x20;

[^2]: We discuss our prioritization considerations under [what-research-to-target](policies-projects-evaluation-workflow/considering-projects/what-research-to-target "mention")\


[^3]: We target these areas (1) because of our current management team's expertise and (2) because these seem particularly in need of The Unjournal's approach. However, we are open to expanding and branching out into other areas.

[^4]: See our FAQ [for-researchers-authors](faq-interaction/for-researchers-authors "mention")

[^5]: We are working to encourage this and make it easy.

[^6]: Opening up the black-box of peer review" and shedding light on _why_ referees and evaluators rate a paper the way they do.&#x20;

[^7]: Research users (policymakers, funders, etc.) learn about the credibility and usefulness of the paper sooner, and in more   detail.

[^8]: Understanding of the research, its connection to other work, its methods, strengths and limitations, its usefulness and policy relevance, etc.

[^9]: Seeing an open discussion of the issues with cutting-edge research helps other researchers make better choices, with better understanding.

[^10]: We can check "when our evaluators rate a paper highly, is it more likely to be published in the top tier journals"?

[^11]: Examples of such tools include Hypothes.is (collaborative annotation) and [Quarto](https://www.quarto.org) and Jupyter notebooks (dynamic documents).
