MetaROR, a new open peer review platform, was launched today at the AIMOS 2024 conference in Canberra, Australia. MetaROR is a joint initiative of AIMOS and RoRI. As a core partner in RoRI, CWTS has been involved in the development of the platform with a couple of colleagues and is proud to contribute to this innovation in academic publishing.
In the past five years, we've witnessed an incredible surge in innovation and energy directed towards metaresearch. Scholars from diverse disciplines have converged to form vibrant communities, pioneering new methodologies and research practice, to understand and improve how scientific findings are produced and disseminated.
This is exciting, but it also presents challenges, particularly in how we communicate and evaluate research.
The challenges we face in the field of metaresearch:
- Siloed Organisation: Research on research communities, such as those in philosophy of science, scientometrics, and innovation studies, often operate independently, with limited cross-disciplinary interaction
- Limited Openness in Research Practices: Much research in our field is shared only in its final state, typically as an article published in a journal, with open science practices like preprinting, open peer review and data sharing uneven across metaresearch communities.
- Limited Accessibility of Literature: Paywalls and publication fees restrict access to metaresearch literature, excluding readers and authors who cannot afford to pay.
- Pressure on Peer Review: Peer review is under pressure, delaying the communication of research outcomes and making it difficult to organise reliable processes for evaluating research results.
- Lack of Community Ownership: Commercial ownership of key metaresearch journals limits communities’ control over review and publication processes.
These problems are also present in other fields, but we believe the field of metaresearch has a particular responsibility to develop and test innovative solutions.
To this end, the Research on Research Institute (RoRI) and the Association for Interdisciplinary Meta-Research and Open Science (AIMOS) are launching MetaROR (MetaResearch Open Review), a new platform for open peer review of metaresearch. Platform services and access are free for authors and readers.
MetaROR is not a traditional scholarly journal, but a platform that operates according to the publish-review-curate model.
Publish
Under MetaROR’s publish-review-curate model, researchers will first publish their work on a preprint server or in a repository–for example, arXiv, bioRxiv, medRxiv, MetaArXiv, OSF Preprints, PsyArXiv SocArXiv, and Zenodo among others–and then submit it to MetaROR.
Review
A MetaROR editor will assign reviewers with appropriate expertise. Review reports and (optionally) reviewer identities will be published on the MetaROR platform and linked to the article under review.
Curate
The MetaROR editor will publish an editorial assessment. Informed by the review reports, the editorial assessment summarises the strengths and weaknesses of the article. However, unlike traditional journal editors, the MetaROR editor won’t make a binary accept/reject decision.
This model will accelerate the communication of scholarly work since peer review will take place after publication rather than before. The model will also increase the value of peer review. Peer review will benefit not only the authors of a work but also the readers, who can use the published review reports to assess the research.
Authors of research reviewed and curated by MetaROR can opt to submit their work for publication in a traditional journal.
To streamline this process, MetaROR is developing partnerships with journals in various metaresearch disciplines. Should authors of reviewed and curated articles choose to submit their work to a journal, they can include the MetaROR review reports and editorial assessment in their submission, enabling journals to rapidly decide whether or not to publish the article.
So far, four journals, Collabra, Quantitative Science Studies, Science as Culture, and Scientometrics, have expressed an interest in partnering with MetaROR, with more to follow.
Browse MetaROR
MetaROR is a community-owned platform run by academics working in diverse metaresearch (or research on research) disciplines. It aims to serve all metaresearch disciplines, including higher education studies, history of science, philosophy of science, science and technology studies, science of science, scientometrics, and sociology of science. You can browse the platform and learn how to submit your work for review below:
A couple of colleagues from CWTS are involved in the MetaROR project through our partnership with RoRI: André Brasil, Wolfgang Kaltenbrunner, and Ludo Waltman. As a new platform for open scholarly communication, MetaROR is closely linked to the work done in the Evaluation & Culture Focal Area at CWTS. Beyond that, CWTS colleagues are involved in editorial roles in several of the journals that have already expressed an interest in partnering with MetaROR.
More information about the platform will be given in an online launch seminar next week Tuesday, 26 November. In this seminar, Kathy Zeiler and Ludo Waltman, the Editors-in-Chief of MetaROR, will introduce the platform and discuss the benefits of MetaROR’s innovative publish-review-curate model. In addition, authors of articles reviewed by MetaROR will share their experiences, and journal editors will reflect on how they expect to collaborate with MetaROR to streamline publication processes. Anyone interested in metaresearch or innovation in academic publishing is welcome to join this seminar!