NY Times distorts peer review

The NY Times has an odd article called "For Science's Gatekeepers, a Credibility Gap". For starters, this article (by a physician named Lawrence Altman) is not that well written. There is very little serious reporting in the article; instead, it's filled with general sentences like this one:

"But many authors have still withheld information for fear that journals would pull their papers for an infraction. Increasingly, journals and authors' institutions also send out news releases ahead of time about a peer-reviewed discovery so that reports from news organizations coincide with a journal's date of issue."

Normally, at this point, a genuine reporter would then cite sources who were interviewed for the article, or go into more detail. Instead, this piece just continues on with more of the same:

"A barrage of news reports can follow. But often the news release is sent without the full paper, so reports may be based only on the spin created by a journal or an institution."

Again, no sources or specific examples are cited.

Besides the fact that this article has no serious reporting, it completely mischaracterizes the peer-review system that is the standard for publishing serious research. Here are some gems from the article:

"The publication process is complex. Many factors can allow error, even fraud, to slip through. They include economic pressures for journals to avoid investigating suspected errors; the desire to avoid displeasing the authors and the experts who review manuscripts; and the fear that angry scientists will withhold the manuscripts that are the lifeline of the journals, putting them out of business. By promoting the sanctity of peer review and using it to justify a number of their actions in recent years, journals have added to their enormous power."


Huh??? "They fear that angry scientists will withold the manuscripts..." I personally know prominent scientists who frequently have papers rejected by top journals; I also personally know editors of journals. The idea that editors accept fraudulent science because they are afraid of angering scientists who will never send them a manuscript again is just total crap. Rejection is a normal part of life for all scientists. Nobody gets all of their papers into Science or JAMA, and fear of authors' anger is not really a major factor in editors' decisions.


Is peer review just part of a cynical game on the part of journals to add 'to their enormous power?" This article tries to make you think so:

"The release of news about scientific and medical findings is among the most tightly managed in country. Journals control when the public learns about findings from taxpayer-supported research by setting dates when the research can be published."

This is absurd - journals are trying to exert tight control over scientific information "by setting dates when the research can be published"???? What happens in reality is this: authors submit a paper, and when it's accepted, it enters the publication pipeline and gets published on a routine timescale. Sometimes a journal will receive two closely related papers in a short period of time; the editors will try to publish both articles back to back in the same issue. In other cases, the findings of a particular study are considered unusually significant, and the manuscript will go through an accelerated publication process (after the paper makes it through peer-review). While a manuscript is under review or awaiting publication, authors frequently present their findings in seminars and conferences, which are generally open to any taxpayer or journalist who cares to attend and listen to a highly technical talk. There is no conspiracy to control what the public learns.

And then there's the paragraph that I quoted before: [I'm not sure what the logical connection is between these two sentences in this paragraph]

"But many authors have still withheld information for fear that journals would pull their papers for an infraction. Increasingly, journals and authors' institutions also send out news releases ahead of time about a peer-reviewed discovery so that reports from news organizations coincide with a journal's date of issue."

It's hard to resist the temptation to just quote bad paragraph after bad paragraph (just go read the article yourself.) It's just so odd that this was considered fit to print by the NY Times.

Here is where I think Altman really goes wrong (other than his attempt to ascribe peer-review's power to a conspiracy of power-hungry journal editors):

Peer-review isn't really set up to catch fraud. And this is not because editors are trying to avoid offending authors, or increase their power over the scientific and medical information available to the public. The real reason is this: peer-reviewers check a manuscript for flaws in reasoning and methodology (did the authors leave out any crucial experiments, was the experimental design subject to unacknowledged error, or did they misinterpret their results?), and they evaluate a paper's scientific merit (they ask, is this a significant achievement in the field?). That's basically it. If authors have falsified their data, reviewers may very well not catch it. Peer-review is still based on trust.

Is scientific fraud a growing problem? Only because the population of scientists is growing. If you have more practicing scientists, you're going to have more fraud. And fraud is caught (always by other scientists) and punished severely. Unlike in politics, business, and even law and medicine, fraud is a complete career-ender in science. Scientists guilty of serious fraud cannot get grants, they lose their academic positions, and can never publish in a reputable journal again. They can no longer be scientists. Fraud is punished more harshly in science than in almost any other profession.


Maybe the NY Times can use a little peer-review for some of their pieces.