Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial
- PMID: 9872878
- PMCID: PMC27670
- DOI: 10.1136/bmj.318.7175.23
Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial
Abstract
Objectives: To examine the effect on peer review of asking reviewers to have their identity revealed to the authors of the paper.
Design: Randomised trial. Consecutive eligible papers were sent to two reviewers who were randomised to have their identity revealed to the authors or to remain anonymous. Editors and authors were blind to the intervention.
Main outcome measures: The quality of the reviews was independently rated by two editors and the corresponding author using a validated instrument. Additional outcomes were the time taken to complete the review and the recommendation regarding publication. A questionnaire survey was undertaken of the authors of a cohort of manuscripts submitted for publication to find out their views on open peer review.
Results: Two editors' assessments were obtained for 113 out of 125 manuscripts, and the corresponding author's assessment was obtained for 105. Reviewers randomised to be asked to be identified were 12% (95% confidence interval 0.2% to 24%) more likely to decline to review than reviewers randomised to remain anonymous (35% v 23%). There was no significant difference in quality (scored on a scale of 1 to 5) between anonymous reviewers (3.06 (SD 0.72)) and identified reviewers (3.09 (0.68)) (P=0.68, 95% confidence interval for difference - 0.19 to 0.12), and no significant difference in the recommendation regarding publication or time taken to review the paper. The editors' quality score for reviews (3.05 (SD 0.70)) was significantly higher than that of authors (2.90 (0.87)) (P<0.005, 95%confidence interval for difference - 0.26 to - 0.03). Most authors were in favour of open peer review.
Conclusions: Asking reviewers to consent to being identified to the author had no important effect on the quality of the review, the recommendation regarding publication, or the time taken to review, but it significantly increased the likelihood of reviewers declining to review.
Comment in
-
Opening up BMJ peer review.BMJ. 1999 Jan 2;318(7175):4-5. doi: 10.1136/bmj.318.7175.4. BMJ. 1999. PMID: 9872861 Free PMC article. No abstract available.
Similar articles
-
Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial.BMJ. 2010 Nov 16;341:c5729. doi: 10.1136/bmj.c5729. BMJ. 2010. PMID: 21081600 Free PMC article. Clinical Trial.
-
Differences in review quality and recommendations for publication between peer reviewers suggested by authors or by editors.JAMA. 2006 Jan 18;295(3):314-7. doi: 10.1001/jama.295.3.314. JAMA. 2006. PMID: 16418467
-
Effects of training on quality of peer review: randomised controlled trial.BMJ. 2004 Mar 20;328(7441):673. doi: 10.1136/bmj.38023.700775.AE. Epub 2004 Mar 2. BMJ. 2004. PMID: 14996698 Free PMC article. Clinical Trial.
-
A primer: peer review process for Advances in Physiology Education.Adv Physiol Educ. 2024 Dec 1;48(4):932-935. doi: 10.1152/advan.00127.2024. Epub 2024 Sep 5. Adv Physiol Educ. 2024. PMID: 39236106 Review.
-
Peer review in medical journals: Beyond quality of reports towards transparency and public scrutiny of the process.Eur J Intern Med. 2016 Jun;31:15-9. doi: 10.1016/j.ejim.2016.04.014. Epub 2016 Apr 26. Eur J Intern Med. 2016. PMID: 27129625 Review.
Cited by
-
Non-financial conflicts of interest in academic grant evaluation: a qualitative study of multiple stakeholders in France.PLoS One. 2012;7(4):e35247. doi: 10.1371/journal.pone.0035247. Epub 2012 Apr 9. PLoS One. 2012. PMID: 22496913 Free PMC article.
-
Welcome to Journal of Foot and Ankle Research: a new open access journal for foot health professionals.J Foot Ankle Res. 2008 Jul 28;1(1):1. doi: 10.1186/1757-1146-1-1. J Foot Ankle Res. 2008. PMID: 18822156 Free PMC article.
-
Peer-review policy and guidelines for Biochemia Medica Journal.Biochem Med (Zagreb). 2014 Oct 15;24(3):321-8. doi: 10.11613/BM.2014.034. eCollection 2014. Biochem Med (Zagreb). 2014. PMID: 25351350 Free PMC article.
-
Peer Review: Publication's Gold Standard.J Adv Pract Oncol. 2012 Mar;3(2):117-22. J Adv Pract Oncol. 2012. PMID: 25059293 Free PMC article. Review. No abstract available.
-
Retrospective analysis of the quality of reports by author-suggested and non-author-suggested reviewers in journals operating on open or single-blind peer review models.BMJ Open. 2015 Sep 29;5(9):e008707. doi: 10.1136/bmjopen-2015-008707. BMJ Open. 2015. PMID: 26423855 Free PMC article.
References
-
- Van Rooyen S, Godlee F, Smith R, Evans S, Black N. The effect of blinding and masking on the quality of peer review: a randomized trial. JAMA. 1998;280:234–237. - PubMed
-
- McNutt RA, Evans AT, Fletcher RH, Fletcher SW. The effects of blinding on the quality of peer review. JAMA. 1990;263:1371–1376. - PubMed
-
- Godlee F, Gale C, Martyn C. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. JAMA. 1998;280:237–240. - PubMed
-
- Rennie D. Commentary on: Fabiato A. Anonymity of reviewers. Cardiovasc Res. 1994;28:1142–1143. - PubMed
-
- Lock S. Commentary on: Fabiato A. Anonymity of reviewers. Cardiovasc Res. 1994;28:1141. - PubMed
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources