Comment on page
- 1.Hosting 'qualitative' evaluation content: A place to host the evaluations (as well as the authors’ responses and the editors' comments), allowing the public to read them in an attractive and convenient way (and perhaps respond to them). Ideally this will also make the quantitative ratings and predictions prominent and connected to the evaluations. This system needs to allow evaluations of any research that is publicly hosted and has a DOI.
- 2.DOIs, Bibliometrics, Google Scholar: We need these evaluations to be visible in "bibliometric systems". They need a DOI, they need to show up in Google scholar and other search tools. The references cited in the evaluations (including the original paper) should also appear in the bibliometric record. Right now 'CrossRef' seems to be the leading system for this.Ideally also...
- 3.Curation and organization: A place to bring together all of the evaluations we have done as the sort of center of our project, to explain it and get positive attention, as well as engage participation and readership.
- 4.'Editorial management': A tool to coordinate our management process (submissions, evaluations, et cetera)
- 5.Hosting and open-analysis of 'quantitative' ratings and meta-data: A place to organize the evaluation data, particularly the quantitative ratings and predictions, in ways that people can analyze and use.Ideally also...
- Ways to enable evaluators and others to do collaborative annotation on pre-prints
- Integrations with other platforms including prediction markets and OSF
- Platforms for people to engage in other ways, perhaps up- and downrating evaluations etc
We have a partially-built system combining Sciety, Kotahi, and Hypothes.is. However, other tools and bespoke work will be needed to achieve some of the goals stated above.
- 1.Ideally also, this would update automatically. We might also want to give some nice analysis and visualization for this (although we could do that in one of the other pages, simply bringing in the data from this place where it’s stored). An API and link to other data archives could be useful.