Tech scoping
Updated see Gdoc on 'Editorial Management Tech needs HERE and embedded at bottom (to do: integrate these discussions)
We are eligible/may be eligible for some nonprofit discounts
Core needs
Hosting 'qualitative' evaluation content: A place to host the evaluations (as well as the authors’ responses and the editors' comments), allowing the public to read them in an attractive and convenient way (and perhaps respond to them). Ideally this will also make the quantitative ratings and predictions prominent and connected to the evaluations. This system needs to allow evaluations of any research that is publicly hosted and has a DOI.
DOIs, Bibliometrics, Google Scholar: We need these evaluations to be visible in "bibliometric systems". They need a DOI, they need to show up in Google scholar and other search tools. The references cited in the evaluations (including the original paper) should also appear in the bibliometric record. Right now 'CrossRef' seems to be the leading system for this.
Curation and organization: A place to bring together all of the evaluations we have done as the sort of center of our project, to explain it and get positive attention, as well as engage participation and readership.
'Editorial management': A tool to coordinate our management process (submissions, evaluations, et cetera)
Hosting and open-analysis of 'quantitative' ratings and meta-data: A place to organize the evaluation data, particularly the quantitative ratings and predictions, in ways that people can analyze and use.
Ideally also...
Some other desirable capabilities
Ways to enable evaluators and others to do collaborative annotation on pre-prints
Integrations with other platforms including prediction markets and OSF
Platforms for people to engage in other ways, perhaps up- and downrating evaluations etc
Further discussion HERE ("Unjournal platform discussion")
We have a partially-built system combining Sciety, Kotahi, and Hypothes.is. However, other tools and bespoke work will be needed to achieve some of the goals stated above.
2023 Update: We have moved mainly to PubPub, at least for hosting evaluation output (see https://unjournal.pubpub.org/)
Partial restatement and update, 2023
\
Ideally also, this would update automatically. We might also want to give some nice analysis and visualization for this (although we could do that in one of the other pages, simply bringing in the data from this place where it’s stored). An API and link to other data archives could be useful.
Last updated