Just Say No to "Wasteful" Reviewer Experiments

In April 27, 2011, Nature published an opinion piece by Prof. Hidde Ploeugh, calling for the "end of the wasteful tyranny of reviewer experiments."

Rather than reviewing what is in front of them, referees often design and demand experiments for what would be better addressed in a follow-up paper. It is also commonplace for reviewers to suggest tests that, even if concluded successfully, do not materially affect conclusions. These are known in the trade as reviewer experiments. The demands seem to increase with the impact factor of a journal, as if referees feel that they need to raise the bar on the journal's behalf.

Ploeugh voiced the many frustrations felt by authors (particularly in biomedical fields) whose PhDs were delayed or who missed out on jobs and promotions because reviewers' requests for additional experiments delayed or prevented acceptance of their articles into high-profile journals.

Almost 2 years have passed since Ploeugh's piece was published, but to many authors the concerns he addressed are still real and relevant.

It is still commonplace for reviewers to "design and demand" additional experiments for authors to conduct before they are willing to recommend an article for final acceptance.  There are a few reasons this practice has continued:

• Submissions, particularly to high Impact Factor journals, continue to increase every year.  This puts greater pressure on journal Editors to be increasingly selective about what they will accept for publication in order to keep their journals' Impact Factors high.
• Reviewers themselves are inundated with constant requests to review.  As a result, they seem to be applying greater scrutiny to the papers they are willing to review.
• The "publish or perish" culture has become more cutthroat in this current scientific environment, leading to more reports of author misconduct and an increase in the number of retractions.  Consequently, editors are more likely to concede to a reviewers who say that more experiments are necessary to substantiate authors' claims.

Ploeugh recommends several ways that this problem can be fixed, but it really comes down to a simple and obvious question that the Editor should pose to the reviewer before making a decision: Are the experiments necessary because the data does not support the conclusion?  If the answer to that question is yes, then the manuscript should be rejected.   

The views and opinions expressed on this site are my own alone and do not represent the views of my employer.