Scientists are under constant pressure to publish their research work. The phrase ‘publish or perish’ is an integral part of a researcher’s life. In fact, apart from its importance to scientific progress and knowledge sharing, research work published in good scientific journals leads to recognition and more funding for future work. However the first publishing step is the peer review process. The peer review system, which has been in place from the early 1700s1, is essentially a critical evaluation of a scientist’s research work by other experts (his/her ‘peers’) from the same field. These experts are referred to as reviewers or referees.
The peer review process has undergone considerable changes over the years mainly to improve its quality. The importance of this system cannot be underestimated. It acts as a ‘quality control’ for scientific research and ensures its credibility. For journals, it is a way to improve and maintain global reputation by publishing only those papers that match their standards. Broadly speaking, three peer-review procedures currently exist:
Types of peer review
Single-blind peer review- This is the traditional approach and is the most common type of peer review process used. In this the reviewer knows the author’s identity but not vice versa. This safeguards the reviewer allowing them to freely comment without any influence from the authors. However, a drawback is that reviewers with a negative or positive personal scientific bias towards the author can potentially influence the chances of publication success. Furthermore, at times, especially if the reviewer is working on the same scientific problem, they can potentially delay feedback, giving them an unfair opportunity to publish their own results first. This is a cause for major concern amongst authors, since the impact of their work is reduced. Another issue is that reviewers may be less critical of publications from senior or respected scientists. An example is the recent discovery that stem cells can be created by dipping blood cells in acid bath. When published in early 2014, the discovery received considerable media attention. However, due to the significance of the discovery, the publication was (and is) subject to immense scientific scrutiny. Possible flaws have been found and the paper is now potentially going to be retracted. Since one of the authors, Teruhiko Wakayama, is a preeminent researcher in cloning, other scientists (and possibly peer reviewers) who were sceptical of the findings, took the work seriously when it was first published. However Wakayama, himself, now doubts the results and has decided to retract the paper, undertake further experiments and re-publish it.2
Double-blind peer review- Here the reviewers’ as well as the authors’ identities are kept anonymous. This is advantageous since reviewers are more likely to give an unbiased evaluation of the work. The reviewer is neither influenced by the seniority of the author nor the quality of previous work: aspects that might create a bias making it easier for the senior author’s publication to get through. Another advantage is that there is no influence of location, for instance publications from countries or organisations that are scientifically more prolific might have an undue advantage. The reviewer is also not influenced by previous controversial work by the same author that can potentially create a negative bias towards the current work. However, for niche areas, and especially if the author is from the same field as the reviewer, the reviewer can usually ‘guess’ the author’s identity. Likewise, when comments are returned to the author, the author can sometimes predict the identity of the reviewer, although this is more challenging. Finally, double-blind peer review also does away with potential gender bias. For instance, although over 20 years old, a 1990 study showed that female reviewers were more likely to accept a publication by a female author than if the same publication was by a male author. There was no such bias from male reviewers.3 A contentious subject, but using the double-blind method does away with this potential bias.
Open peer review- This is a recent approach where the identities of both the reviewers and the authors are known to each other, and to the general public. An open peer review also usually involves a larger group of reviewers who comment on a manuscript that is already published, usually online. The rationale being that it compels the reviewers to give constructive input to improve research quality without being vague on experimental suggestions. It also potentially initiates constructive dialogue between the reviewers and the authors. However, a major drawback is that critical feedback may be withheld for fear of retribution by the authors. Also junior scientists may not be very critical of senior scientists’ work.
Which of the above approaches is fairer? Arguably the double-blind method is the fairest, but what is your take?
Scientific fraud can slip through the peer review net!
A fundamental issue in the peer review process is its inability to identify scientific misconduct and fraud. Misconduct or fraud is exposed when others cannot replicate published experimental results, especially those that claim radical innovations or breakthroughs that challenge the accepted norm. Such publications naturally receive immense scientific scrutiny even after they have gone through the journal’s peer review process. A classic example is the Bubble Fusion claim published in Science in 20024. When the paper came out, it was heralded as a breakthrough by creating a low cost, desktop source of unlimited energy. However, due to the implications of this it was subject to extensive analysis by the scientific community. No one from the scientific peer group was able to replicate the results, and the publication and its lead author were discredited. Thus even though the publication was likely subjected to extensive peer review prior to publication, since it was radical and was also submitted to a leading journal, it highlights a fundamental flaw: the peer review process is not foolproof.
A new peer review strategy
The most obvious questions that arise are: Is there a way to make the system flawless? How do you put a check on the ‘quality control’ system itself? Should there be a standardised process across all journals and/or should journals that are high-impact (read as responsible for publishing major scientific breakthroughs), follow a different peer review approach? Should an alternate method be used for publications that challenge the status quo? For instance, an experimental peer review i.e. journals ask reviewers to physically visit laboratories where experiments were performed, audit the experimental procedure, and confirm the result! Although such an approach is radical, it is possibly one way to guarantee that ground-breaking results are indeed that: Ground-breaking!
What would you do as a scientist???
References
1. Benos, D. J. et al. The ups and downs of peer review. Adv. Physiol. Educ. 31, 145–152 (2007).
3. Lloyd, M. E. Gender factors in reviewer recommendations for manuscript publication. J. Appl. Behav. Anal. 23, 539–543 (1990).
4. Taleyarkhan, R. P. et al. Evidence for nuclear emissions during acoustic cavitation. Science 295, 1868–1873 (2002).
