Thursday 30 May 2013

Peer review fail, like finding Mummy's been sleeping with the Milkman.

Peer review of scientific journals is corrupt if it isn't blind
and the gatekeepers of knowledge go tribal
to boost the papers of friends and high-status players.

Check this gem discovered by smalldeadanimals.com, published by the Cambridge University Press, abstracting work by Peters and Ceci.

“The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we selected 12 already published research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices. (my bolding). With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated”. 
I love to find out things and feel broken-hearted that researchers I want to trust are untrustworthy. It's like learning Mummy has been sleeping with the Milkman.  (I'm old enough to remember milkmen going door to door.)

The rejection rate reported is so high we are losing good ideas. Self-publishing on the internet may work-around this.

The comely figure of justice with a blindfold and scales is passé for some. I like her just fine.

No comments:

Post a Comment