Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2011;6(11):e26895.
doi: 10.1371/journal.pone.0026895. Epub 2011 Nov 9.

Cooperation between referees and authors increases peer review accuracy

Jeffrey T Leek  1 , Margaret A Taub, Fernando J Pineda
Affiliations

Cooperation between referees and authors increases peer review accuracy

Jeffrey T Leek et al. PLoS One. 2011.

Abstract

Peer review is fundamentally a cooperative process between scientists in a community who agree to review each other's work in an unbiased fashion. Peer review is the foundation for decisions concerning publication in journals, awarding of grants, and academic promotion. Here we perform a laboratory study of open and closed peer review based on an online game. We show that when reviewer behavior was made public under open review, reviewers were rewarded for refereeing and formed significantly more cooperative interactions (13% increase in cooperation, P = 0.018). We also show that referees and authors who participated in cooperative interactions had an 11% higher reviewing accuracy rate (P = 0.016). Our results suggest that increasing cooperation in the peer review process can lead to a decreased risk of reviewing errors.

PubMed Disclaimer

Conflict of interest statement

Competing Interests: The authors have declared that no competing interests exist.

Figures

Figure 1
Figure 1. Open versus closed peer review systems for the peer review game.
Under the closed system of peer review (left column), reviewers know the identity of problem solvers, but problem solvers do not know the identity of the reviewers. Public information is limited to the number of accepted solutions for each player. Under the open system of peer review (right column) solvers and reviewers are known to each other, and both the number of accepted solutions and accepted reviews for each player are known publicly.
Figure 2
Figure 2. Open peer reviewers spend a greater proportion of their time reviewing.
Each panel is a plot of the cumulative proportion of time each individual spends solving problems during the experiment over the course of the game. Under closed peer review, individuals spend a greater proportion of their time solving (top row). In two experiments (Closed Experiment 1, Closed Experiment 2), an individual spent almost 100% of their time solving problems. Under open peer review, individuals spent a smaller proportion of their time solving problems and a greater proportion of their time reviewing problems (bottom row).
Figure 3
Figure 3. Open reviewers are more cooperative than closed reviewers.
Each panel shows the cooperation network for one of the peer review experiments. The thickness of the line indicates the amount of interaction and color indicates the type of interaction. Cooperation (blue) is defined as above average probability of both players accepting each others solutions. Obstruction (red) is defined as below average probability of both players accepting each others solutions. Under closed review (top row) there is less cooperation between players than under open review (bottom row).
Figure 4
Figure 4. The task selection screens for the Peer Review Game.
Task selection under the (a) closed and (b) open modes. In each case a player may elect to solve or review a problem. In the open peer review mode, players know the identity of the players reviewing their solutions.
Figure 5
Figure 5. The problem solving screen for the Peer Review Game.
The problem solving screen is the same for both versions of the game.
Figure 6
Figure 6. The reviewing screen for the Peer Review Game.
The reviewing screen is the same for both versions of the game.
Figure 7
Figure 7. The public information screens for the Peer Review Game.
Public information under the (a) closed and (b) open modes. In each case the number of solutions each player has had accepted are displayed. In the open review system, the number of solutions reviewed and accepted by each player is also displayed.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Fabiato A. Anonymity of reviewers. Cardiovasc Res. 1994;28:1134–1139. - PubMed
    1. Wenneras C, Wold A. Nepotism and sexism in peer-review. Nature. 1997;387:341–343. - PubMed
    1. Smith R. Opening up BMJ peer review. BMJ. 1999;318:4–5. - PMC - PubMed
    1. Mietchen D. Peer reviews: make them public. Nature. 2011;473:452. - PubMed
    1. Pulverer B. Transparency showcases strength of peer review. Nature. 2010;468:29–31. - PubMed