Defining Manuscript Peer Review: An International Delphi Consensus Study
Abstract
Abstract Objectives To conduct a Delphi process to inform an international, consensus definition of manuscript peer review. Design Three-round Delphi consensus study including two surveys and one virtual consensus meeting. Peer review features identified through a prior scoping review were rated on a 5-point Likert scale in Rounds 1 and 2 and voted on using include, exclude, and abstain options in Round 3. Consensus was defined as ≥80% agreement. Setting International experts representing interest holders affected by manuscript peer review across the fields of biomedical, health, and social sciences, economics, and healthcare-related industries. Participants Round 1 participants were purposively recruited via email from corresponding authors of articles included in the preceding scoping review. Round 2 participants were those who completed Round 1. Round 3 participants were purposely selected to address gaps in representation from earlier rounds, with attention to diversity of perspectives and equity-relevant characteristics informed by PROGRESS-Plus (e.g., geographic location, gender, race/ethnicity, occupation, and socioeconomic context). Participants were required to communicate in English. Main outcome measures Consensus on the terminology, scope, characteristics, functions, processes, and reviewer characteristics that should inform a definition of manuscript peer review. Results Across the three rounds (N=143, 109, and 16 participants, respectively), 44 peer review features across six domains (Characteristics, Function, Motivation, Assessment, Process, and Challenges) and six peer reviewer characteristics met the consensus threshold for inclusion. Consensus was also reached on terminology to describe manuscript peer review and the peer review workflow. Results were synthesized into a final conceptualization of manuscript peer review, consisting of one definition and eight pillars.The consensus definition is: Manuscript peer review is a structured process by which human reviewers assess scholarly manuscripts, with the aim of improving the quality and trustworthiness of published research in line with norms of reciprocity to the research community. The eight pillars characterize manuscript peer review and are as follows: Principled, Constructive, Timely, Transparent, Professional, Quality, Confidential, and Pluralistic. Conclusions This three-round international Delphi established a consensus-based, normative conceptualization of manuscript peer review as a distinct, human-led process to improve the quality and trustworthiness of scholarly research. The resulting definition and eight pillars may provide a shared foundation for research, evaluation, education, and improvement across publishing contexts while allowing operational features to remain adaptable.
Authors 29
-
Affiliation as printed
Western University;
-
Zimmer Biomet (Netherlands) · Ludwig-Maximilians-Universität München
Affiliation as printed
Ludwig Maximilian University of Munich Institute for Medical Information Processing Biometry and Epidemiology;
-
Universidade Estadual de Campinas (UNICAMP)
Affiliation as printed
Universidade Estadual de Campinas, Campinas;
-
Affiliation as printed
Hassanah Consulting;
-
Affiliation as printed
PCORI;
-
Affiliation as printed
AI Rights Initiative;
-
Affiliation as printed
Eotvos Lorand University;
-
Affiliation as printed
University of Ottawa;
-
Universidade Federal do Rio de Janeiro
Affiliation as printed
Universidade Federal do Rio de Janeiro;
-
Affiliation as printed
Boston University;
-
Affiliation as printed
University of Geneva;
-
Affiliation as printed
UK Research and Innovation;
-
Affiliation as printed
American Medical Association;
-
Fundación Universitaria del Área Andina
Affiliation as printed
Fundacion Universitaria del Area Andina;
-
University of South Carolina System
Affiliation as printed
University of South Carolina System;
-
Bangalore Medical College and Research Institute
Affiliation as printed
Karnataka Medical College and Research Institute;
-
Universidad Nacional Autónoma de México
Affiliation as printed
National Autonomous University of Mexico;
-
Affiliation as printed
Health Service Executive;
-
Sanjay Gandhi Post Graduate Institute of Medical Sciences
Affiliation as printed
Gandhi Postgraduate Institute of Medical Sciences;
-
University of the West Indies · University of the West Indies · University of the West Indies · University of the West Indies
Affiliation as printed
The University of the West Indies;
-
Affiliation as printed
Universidad del Desarrollo;
-
Affiliation as printed
King Abdulaziz University;
-
Affiliation as printed
Rashid Latif Khan University Medical and Dental College;
-
Ottawa Hospital Research Institute
Affiliation as printed
Ottawa Hospital Research Institute;
-
Affiliation as printed
Center of Research in Epidemilogy and Statistics Sorbone Paris;
-
Affiliation as printed
University of Toronto;
-
Affiliation as printed
Hospital for Sick Children;
-
Ludo Waltman Aachen
Affiliation as printed
Leiden University;
-
David Moher corresponding
Canadian Institutes of Health Research · Ottawa Hospital
Affiliation as printed
Ottawa Hospital Research Institutes