Preprints are proving themselves to be a powerful point of leverage which are facilitating a much greater fluidity of scholarly communication by surfacing its traces – and quite simply for researchers the creative pleasure of making better tools. The recent announcement from eLife to only review preprints and focus on publishing reviews – as a model of ‘publish, then review’ goes to illustrate the role of preprints and open peer review in a more connected Open Science research cycle.
The PreprintSprint was organised by ASAPbio over a twenty day period ending with presentations and awards on the 3rd December 2020. Twenty-one projects took part in the design sprint which kicked off on the 13th November with an intensive round of breakout sessions. (NB: see the full slide deck of all the projects).
There were a number of awards made by a panel of judges, with an additional ‘people’s choice’ in addition, at the final closing presentations of the design sprint on the 3rd December.
- Best in show: ‘Transforming Peer Review through Mentorship and Community Engagement’, @PREreview_
- In two award categories – People’s choice, and Increasing representation: ‘Building capacity for preprint peer review and curation in Africa’, @tccafrica & @AfricArxiv
- Project development: ‘Early Evidence Base’, @EMBOPress
- Promoting constructiveness: ‘Take a Penny, Leave a Penny’, @researchsquare
All the project contributions to the design sprint made valuable contributions to the field. GenR picked a selection from the twenty-one that represented the Open Science interests of connected and interoperable systems.
Piloting peer review overlay services on a distributed network of preprint servers and repositories
Kathleen Shearer @KathleeShearer Executive Director of COAR (Confederation of Open Access Repositories) led a team effort which has been outlined fully here. With the market expanding for review platforms, browser plugins, etc., this is a vital technology strand – to bring reviews to the reader and vice versa – to bring reviewers to the research.
‘…to develop a standard technical approach to bi-directional communications between preprint services (and other types of repositories) and peer review services, that can scale beyond individual initiatives.’
The project is a reference implementation and uses the W3C Linked Data Notifications (LDN) protocol to message about status and requests of review, such as – review requested, new version, etc. LDN has been advocated by team member Herbert Van de Sompel as a way to de-silo the net and has been used by Sarven Capadisli as a key component of the autonomous publishing implementation Dokieli.
Peeriodicals is from the makers of PubPeer browser plugin and was represented by Brandon Stell and Boris Barbour. Peeriodicals allow an individual or group to curate their own journal – like a collection in Zotero, or on other Open Access platforms such as F1000.
At the PreprintSprint the project was looking for uptake by the scientific community. An example journal that shows off the platform is Theoretical neuroscience. The project starts to come alive when you notice the embedded Pubpeer reviews, which might well be coming from a variety of sources from where the original article was posted on say a Preprint server. An interesting set of use cases for Peeriodicals might well be on the edge of academia, or in information services: library subject specialists profiling work, in new epistemological fields with no existing journal, or for those often not catered for by collection management systems – the citizen scientist.
I Owe The Academy: Portable Tokens for Open Peer Review #IOTA
Philip N. Cohen and Micah Altman took an approach of market design for peer review and outlined their ideas in the following white paper ‘Market for Peer Review: How it has failed, and how to fix it’. Two motivations are put forward for the idea of a ‘token’ that a reviewer would receive for their efforts – firstly that across the board soliciting quality reviews are hard to come by, and secondly, that review systems outside of journals can’t attract reviewers. A set of market place interactions are mapped out: for reviewers gaining a token by carrying out a review, or declining the review and passing on the review slot; for editors handing out tokens; and for the article readers being able to see an audit trail of the review.
The project is at a readiness level of conceptualizing the idea and moving it towards being an MVP. In the white paper a number of criteria and checklists have been brought together for the purpose of incrementally designing and evaluating such a market and #IOTA token system.
‘Its goal is to establish a peer review exchange service that will harness real peer reviews and also generate systematic knowledge about peer review’ – #IOTA white paper
Preprint review and curation by content type
Daniel Mietchen @EvoMRI has taken on a cutting edge challenge of looking at the questions of how to review research material associated with a preprint which has varied content types, is versioned, and requires software to run code. Daniel folded this investigation into a #100DaysOfJupyter that he is currently embarked on – days #60 below give a way into some of the work. The use of a Jupyter Notebook gives an environment where data and computational content can at least be addressed and interestingly here PAWS the Wikimedia deployment of Jupyter Notebooks is being used.
In conclusion it is worth pointing out that all of the projects were looking for collaborations and to work with partners. The PreprintSprint is situated with an ongoing body for work organised by ASAPbio of advocating, supporting, and driving forward the twin threads of preprints and open peer review. Some example resources from ASAPbio are the ReimagineReview which tracks innovative review services and Review Commons a review repository run in partnership with EMBO.
Generation Research. “ASAPBio #PreprintSprint Presentations.” Generation Research, 2020. https://doi.org/10.25815/BC5T-1178.