Peer Review and Careers

Last spring I had the good fortune to meet one of my cousins for dinner in Philadelphia. She is an economist trained and working within the Italian university system, but she was working in the U.S. at the time. Over dinner we did what any two academics from different continents are likely to do — we compared notes. Inevitably, our conversation turned to the academic publishing system. Her knowledge is far more international than mine, as she has published in the journals of a variety of languages, based in several different countries.

“It’s so frustrating how long it takes to get things into print,” she said.

“How long is ‘long’?” I asked.

“Months!” she exclaimed.

I literally laughed in my cousin’s face. She was subsequently appalled by some of my horror stories about publishing in American journals, and shared a few stories of her own.

I have no doubt that the humanities disciplines are, on the whole, the worst offenders when it comes to how long it takes to generate reader reports, and to move an article from an initial submission to a finished, published product. If it can take two years to publish humanities research in some traditional, print-based journals — and I’m talking articles here, not books — that lag makes it harder than ever to defend the project of humanities disciplines. How, and I’m lecturing directly to fellow humanists here, can we defend the importance of our disciplines if our publishing processes, so fundamental to knowledge production, are so lumbering, and (generally) so lacking in accountability? But I digress, perhaps.

The sciences, and particularly the experimental fields, are acutely aware of the speed with which knowledge evolves. Particularly within the hard sciences, new models of publishing are developing in order to get new research into circulation, in addition to the persistence of more traditional and slower-moving publishing practices.

Slow publishing times, unfortunately, are not the only problem that plagues the publication process. Regardless of which discipline one works from within, any teacher-scholar with publishing responsibilities quickly encounters some of the maddeningly inconsistent, and even irresponsible, behavior that characterizes some editors’ and peer reviewers’ practices. In particular, peer reviewing articles for publication, as far as I can tell, is one of the enigmatic black boxes of academe. Stuff happens in there, but nobody can really say with certainty what, at least not in general terms. Rarely are professionals in any field trained in how to perform a blind peer review, even though the activity is one of the foundations upon which knowledge production within the modern academy rests. Double-blind peer review, despite recent, limited, and exciting trials of alternative peer review processes, remains the overwhelming standard for how scholarship is vetted for publication.

Journal editors typically at least receive a degree of mentorship, working up the editorial ranks. Peer reviewers, however, should also be mentored, and receive feedback on their own work from experienced editors. However, because the work of both editing and peer reviewing usually falls into the nebulous category of “service,” we are often happy if the work gets done at all, and rarely does a journal or editor have the time and resources to offer such feedback. Unfortunately, very few scholars in any field are explicitly trained in how to edit or how to conduct peer reviews.

Here are a couple of suggestions for editorial and peer reviewing practices, based on anecdotal experience. First, some practices to avoid. Common editorial lapses include:

Failure to provide editorial direction: Early in my career I received a set of peer reviews that conveyed very different messages. The reviews took six months to arrive. The first review suggested that I revise and resubmit, and elaborated on what issues needed to be addressed in order for the revisions to merit acceptance — a perfectly straightforward and helpful review. The second review recommended rejecting my article, and explained why. Also a perfectly straightforward review, and well within the reviewer’s rights, and professionally articulated. The problem was with the editor who conveyed the reviews to me. She suggested that I revise and resubmit my article. The problem was that the two reviews were so in conflict that, as an author, I was at a loss for how to proceed.

Clearly a reviewer who had rejected the article outright would not be receptive to revisions. Was the editor implying that she was of a mind with the first reviewer? How did she recommend reconciling the criticisms of the rejection, if at all? It is the editor’s responsibility to frame the process, and help the author to know how to proceed. Given that it had taken six months to receive these initial reviews, and that the editor had provided no direction, I simply pulled my submission.  

I recently had an even more pronounced version of this same dilemma. One reader report came back recommending publication, and the other recommended outright rejection of my article. The editor offered no advice for how to reconcile the dramatically different two reviews when inviting me to resubmit, a serious lapse, in my view, of one’s editorial obligations. We’ll see how that one turns out. Simply, editors are there to provide active direction to both authors and reviewers.

The Desk Rejection: The desk rejection, or the editor’s decision not to send an article out for review, is not itself a problem. It is perfectly acceptable for a journal editor to decide that a submitted manuscript does not “fit” the journal or meet the journal’s standards and to decline to send the manuscript out for peer review. Such vetting is, indeed, part of the editor’s (or editorial team’s) job.

However, what’s unacceptable is when it takes four or six months for an author to receive word of the desk rejection from the editorial staff. I have experienced a desk rejection six months after submission myself, and have heard of it from many others. Editorial teams should be reviewing manuscripts frequently enough to either send a manuscript out for review or make the desk rejection within a month of the manuscript’s submission, at the latest. Because journals forbid simultaneous submissions, the time between when a manuscript is submitted until it is rejected is unrecoverable lost time for the author. And, in the case of a desk rejection, the rejected author does not even receive the benefit of peer comments on the manuscript. Desk rejections, when they are appropriate, need to be made as speedily as possible.

The Bad Faith Resubmission: Unfortunately, not all invitations to resubmit an article are made in good faith. Sometimes the offer to resubmit has its origins in the editor’s inability to make a decision. Shockingly, I have even heard (on very good authority) of editors inviting authors to resubmit, with the intention of later rejecting the article. I once had an editor tell me that, while he thought the subject of my submission was important, he and the reviewers were at a loss for how to advise me to remedy several problems (the article had already been revised and resubmitted once), and so he was not inviting me to resubmit again.

While this was deeply disappointing to me, I think it was a highly ethical and responsible move on the editor’s part. He refused to waste my time or the journal’s by inviting me to make a resubmission of an article that he expected would ultimately be rejected. It allowed me to rework the article dramatically for publication in a different venue. Offers to resubmit should always be in good faith and be accompanied by an appropriate amount of editorial direction for what needs to be addressed before an article can be accepted.

Lack of Organization: Nothing tanks a journal like a disorganized, information-losing, deadline-missing editorial team. Nothing. It is bad for authors, bad for reviewers, bad for the journal, and bad for the discipline.

Best Practices

A different set of best practices governs the process of conducting peer reviews. Reviewing is a serious obligation. Scholars who have published their way to tenure have an obligation to contribute peer reviews when their expertise is appropriate and sought out by a journal editor. Having benefited from the double-blind peer review system as scholars, they must have keep the system moving forward.

However, agreeing to review a manuscript does not in and of itself fulfill the obligation. Anonymity, just as it does in plenty of other forums, sometimes breeds particularly noxious behavior on the part of blind reviewers. The rudeness that I have encountered in anonymous reader reports exceeds the rudeness I have encountered in all other forums inhabited by academics — and that’s saying something.  Katrina Gulliver recently wrote about some problems accompanying the peer review process, and I think her article is spot-on, but I have a few things to add.

Here are some best practices for peer reviewers:

• Meet the editor’s deadline for submitting your review. If you can’t complete your work in a timely manner, don’t accept it in the first place. If you anticipate missing the editor’s deadline, at least give her a heads-up.
• Be specific in both your advice and your objections. Refer to specific places and moments within the text under review.
• Be cordial. Any practicing teacher-scholar ought to be able to offer a rigorous critique without rudeness or personalizing the process. Assume goodwill on the part of the anonymous author, and return it. Indignation is rarely an appropriate feature of a peer review.
• Suppress your own ego. Is it really essential that the author cite your work? It might be, and that’s fine, but it might also be the case that any number of citations would be appropriate. Reviewing isn’t an opportunity to pat yourself on the back.
• Be current. Nothing is more vexing than to have a reviewer insist that you cite out-of-date or even debunked work because he himself has not kept up with relevant literature. But it happens all the time.
• Be transparent with the editor. If you have sussed out who the author of an article is, or have something at stake in whether or not the article is accepted, that’s information that the editor needs. Share it. You may still be able to conduct the review, but transparency is a great purifier.  

Experiments with alternatives to the traditional peer review system are ongoing and exciting. But until alternative models are proven and emerge more broadly, we have an obligation to ourselves and our disciplines to engage in professional editorial and review practices within the traditional peer review system. We need to police our own behaviors, and in particular senior scholars, who conduct the bulk of editing and reviewing, need to hold journals, editors, and reviewers to high standards, lauding those who do professional work and installing them as mentors when appropriate, and calling out those unprofessional practices that jeopardize both knowledge production and careers.