Peer Review Failure?

The editors of Proceedings of the National Academies of Sciences insist that all of their articles are subject to peer review. But -- at least until next year -- some peer reviewers are more equal than others, and some authors have their peer reviewers selected by advocates for their work.

This week, the journal is announcing that it is abandoning the two-tiered system in July. At that point, all articles will go through a more traditional process in which submissions that pass an initial review will be sent to anonymous peer reviewers who are experts in appropriate fields. The system of "communicated submissions" -- in which National Academies members can set up their own peer review process for articles they endorse -- will end.

The change has been in the works for months, and the idea has been discussed longer than that. But the announcement comes at a time when the journal is under intense criticism for an article published last month -- via the route in which academy members can organize their own peer review panels -- claiming that caterpillars and butterflies do not have the same evolutionary history. Rather than viewing the butterfly and caterpillar as two life stages, the article views them as evidence of some sort of lasting mistake from a butterfly-like being accidentally mating with a worm at some point in the distant past.

Jonathan Lifland, a spokesman for the journal, said Sunday that the editorial board is reviewing the article to consider whether any action is appropriate. "We are aware of the situation and we are concerned about anything that could reflect poorly on the National Academies and the journal," he said. "The reaction we have seen from scientists has been what you would likely expect. The editorial group at PNAS is studying how to handle the situation."

Any decision on a retraction, Lifland said, would have to come at a formal meeting of the editorial board.

"Worst paper of the year?" was the title of a post on the blog Why Evolution Is True, by Jerry A. Coyne, professor of ecology and evolution at the University of Chicago. He wrote that the theories in the article were "bizarre and completely unsupported," stressing the lack of evidence to back up the hypothesis. Scientific American's coverage was headlined, drawing on a comparison from a Duke University biologist, Fred Nijhout: "National Academy as National Enquirer? PNAS Publishes Theory That Caterpillars Originated from Interspecies Sex."

British scientists are also expressing shock that the article (by one of their own) was published in a highly respected journal. The Times Higher on Sunday quoted Max Telford, a zoologist at University College London, as saying of the article: "There is no science in it. I don’t think it could possibly have got through on a normal peer-review process.... It is unique to PNAS that they have got this backdoor way of getting things through.”

The author of the paper is Donald I. Williamson, a retired professor at the University of Liverpool, and it was "communicated" to the PNAS by Lynn Margulis, a professor at the University of Massachusetts at Amherst, who as a National Academies member has the right to conduct her own peer review for articles she is referring. Williamson and Margulis could not be reached Sunday. But Williamson told the Times Higher that they are friends. He stood by the work, but acknowledged that the paper had been rejected by seven other journals before PNAS published it.

Lifland, the spokesman for the journal, while acknowledging the furor over the article, said that various reports about how the process "bypassed" peer review were incorrect. There are different tracks for the two systems, he said, but all "communicated" articles must be accompanied by peer review reports. The difference is that the NAS members pick those to review the articles that they are in effect sponsoring for publication.

According to an editorial that is appearing in PNAS this week, for many years, these "communicated" articles made up most of the work published in the journal. It was only in 1995 that the process of submitting directly to the journal -- and having its team identify peer reviewers -- was created, the editorial says. Since then, however, the "direct" submission approach has gained in popularity and now accounts for the vast majority of submissions, which led the editorial board (with support from a poll of academy members) to eliminate the "communicated" system.

At the same time, National Academy members will still have the right to look after for some submissions. To encourage work that academy members view as "exceptional, but out of the mainstream," those submitting papers can ask for a review by a particular academy member, who would be a "pre-arranged editor" involved in the peer review process. But the editorial also notes a check on this system: Before such editors can be involved, the journal's staff must "confirm" that they have "the requisite expertise and the paper is appropriate for publication in PNAS."