Friday, June 14, 2013

The Death of the Single Pass Peer Review

One key aspect of peer review is to ensure that the paper is scientifically sound. The publishing system therefore asks peers to review the paper and test that. Part of this is to see if relevant literature is cited, backing up key finding around the addressed hypothesis in the paper. After all, the research must be novel.

However, there is so much literature nowadays in so many different journals, a phenomenon initiated by the large publishing industries, that it has become hard to track things. No worries, these same industries have come up with tools to handle that. A second problem is that research has become that interdisciplinary that even focusing on  a few journals of which you can even read the papers easily is hard.

Consequently, it is easy to miss even key papers. He who reads all relevant, important literature cast the first tweet. Thus, the peer reviewers jump in, and make you aware if you missed something critical.

Now, these peers actually have the same issue. Worse, because it is hard to find good reviewers, we settle with post-docs and peers not specialist in the topic of the paper. Of course, we'll find plenty of reasons why this is good. But the bottom line is, that even peer review fails to address all critical points in the manuscript. And, as a result, we can all point to literature in the glossies where important aspects have been neglected (he who never found a "high impact" paper without flaws, cast the first post), and an increase in the number of retractions recently.

Therefore, what the publishing community needs is to admit that the current approach is no longer sufficient. It worked well, for some 40-50 years, we did not need it before then, and it no longer scales with the output. I am not implying that peer review is bad, but the current single pass peer review implementation is.

Instead, I call on all publishers to step away from the current implementation, and adopt a multistep peer review process. Possible approaches include:

  1. only accept papers that have appeared in a pre-print server
  2. implement a two-pass system, with a quick first review scanning if key literature has been discussed, just covering the introduction
  3. open peer review
I know that some of these solutions are being experimented with. All I ask is for publishers to strongly support this, and demand from editors and authors to do the same.

Why? Because your journal quality will actually improve.

No comments:

Post a Comment