Economics survey (Charness et al.)

Improving peer review in economics: Charness et al. project and survey

We designed and disseminated a survey taken by over 1,400 economists in order to (i) understand their experiences with peer review and (ii) collect opinions about potential proposals to improve the system.
...
We reviewed the existing literature about peer review, drawing on sources from inside and outside of economics. ... We then built a (non-comprehensive) themed bibliography,
... we took the additional step of preparing a list of over 160 proposals.

Other peer-review models Our current peer-review system relies on the feedback of a limited number of ad-hoc referees, given after a full manuscript was produced. We consider several changes that could be made to this model, including:
Post-publication peer review: Submissions could be published immediately and then subjected to peer review, or they could be subject to continued evaluation at the conclusion of the standard peer-review process.
Peer review of registered reports: Empirical papers could be conditionally accepted before the results are known, based on their research question and design. A limited number of journals have started to offer publication tracks for registered reports.
Crowdsourced peer review and prediction markets: Rather than relying on a small number of referees, the wisdom of crowds could be leveraged to provide assessments of a manuscript's merits.
Non-economists and non-academics as referees: Besides enlarging the size of the pool of referees who assess a paper, the diversity of the pool could be increased by seeking the opinion of researchers from other disciplines or non-academics, such as policy makers.
Collaborative peer review platforms: Communication between authors, reviewers, and editors could be made more interactive, with the implementation of new channels for real-time discussion. Collaborative platforms could also be set up to solicit feedback before journal submission occurs.

PreviousOther non-journal evaluation NextManagement details [mostly moved to Coda]

Last updated 12 months ago

Improving peer review in economics: Charness et al. project and survey

We designed and disseminated a survey taken by over 1,400 economists in order to (i) understand their experiences with peer review and (ii) collect opinions about potential proposals to improve the system.

...

We reviewed the existing literature about peer review, drawing on sources from inside and outside of economics. ... We then built a (non-comprehensive) themed bibliography,

... we took the additional step of preparing a list of over 160 proposals.

Other peer-review models Our current peer-review system relies on the feedback of a limited number of ad-hoc referees, given after a full manuscript was produced. We consider several changes that could be made to this model, including:

Post-publication peer review: Submissions could be published immediately and then subjected to peer review, or they could be subject to continued evaluation at the conclusion of the standard peer-review process.
Peer review of registered reports: Empirical papers could be conditionally accepted before the results are known, based on their research question and design. A limited number of journals have started to offer publication tracks for registered reports.
Crowdsourced peer review and prediction markets: Rather than relying on a small number of referees, the wisdom of crowds could be leveraged to provide assessments of a manuscript's merits.
Non-economists and non-academics as referees: Besides enlarging the size of the pool of referees who assess a paper, the diversity of the pool could be increased by seeking the opinion of researchers from other disciplines or non-academics, such as policy makers.
Collaborative peer review platforms: Communication between authors, reviewers, and editors could be made more interactive, with the implementation of new channels for real-time discussion. Collaborative platforms could also be set up to solicit feedback before journal submission occurs.