Pilot: Setting up platforms

Set up the basic platforms for posting and administering reviews and evaluations and offering curated links and categorizations of papers and projects.

Progress reports

Update 7 Sep 2022, partial update 22 Dec 2022

• We are setting up processes and forms in Kotahi

  • Submissions form is pretty useable (but imperfect, e.g., we need to ask people to (click 'submit a URL instead' on page one)

• Evaluations form: using a Gdoc for now, trying out Airtable, Qualtrics and other solutions, aiming to integrate it into Kotahi

• See Mapping evaluation workflow for how projects will enter, be evaluated, and 'output'

• We will outline specific requests for developers\

• Sciety group set up with 'Hypothes.is feed'; working on processing first evaluations\

Submission, evaluation and management platform Kotahi: submit/eval/mgmt (may be phasing out?)

7 Feb 2023

• Set up Kotahi page HERE

• Configured it for submission and management

• Mainly configured for evaluation but it needs bespoke configuration to be efficient and easy for evaluators, particular for the quantitative ratings and predictions. Thus we are using Google Docs (or cryptpads) for the pilot. Will configure Kotahi with further funds.

Sciety group (curated evaluations and research)

Evaluations are curated in our Sciety.org group, which integrates these with the publicly-hosted research.

7 Feb 2023: We are working on

• The best ways to get evaluations from "submissions on Kotahi" into Sciety,

• ... with the curated link to the publicly-hosted papers (or projects) on a range of platforms, including NBER

• Ways to get DOIs for each evaluation and author response

• Ways to integrate evaluation details as 'collaborative annotations' (with hypothes.is) into the hosted papers

(We currently use a hypothes.is workaround to have this feed into Sciety so these show up as ‘evaluated pre-prints’ in their public database, gaining a DOI.

Notes, exploring the platform.