The Case for Open Review

Most people assume that medievalists like me have no interest or investment in new forms of books and publication processes. Nothing could be farther from the truth. While some aspects of digital writing -- especially increased speed, scale and access -- are undoubtedly new, other central features of blogging, wiki and social media platforms are not just old, but so old that they had become virtually obsolete prior to the invention of the computer. In fact, we can say they are newly medieval -- a recognition that has important implications for open communications in peer review.

When we describe an encyclopedia that operates as an unfinished accumulation of information about the world, in which multiple texts are compiled, abbreviated, juxtaposed, revised and recategorized by multiple (often) anonymous contributors over time, we are not just describing Wikipedia. We are describing the miscellaneous and assembled nature of most medieval collections, such as the bestiary, the florilegium and the chronicle. When we describe the expansion of a social network through written correspondence, we are not just describing friending on Facebook or following on Twitter. We are describing the medieval art of letter writing that flourished among notaries and bureaucrats, encouraging writers to share words with friends.

And when we describe a culture of commentary, in which the proliferation of comments upon a text or issue usurp the primacy of the text or issue itself, we are not just describing blog rants on the click-bait article of the day. We are describing the thriving industry of medieval commentaries on classical philosophy, biblical interpretation and legal codes. While all of these practices have continued to exist in various forms throughout the high age of print, they have achieved a prominence today that they have not experienced since the Middle Ages.

Among those many commonalities, the prevalence of commentary reflects a particular disposition toward writing and reading that is at the core of open-access movements. Advocates for open access have been working tirelessly to make scholarly work freely available online without most copyright and licensing restrictions, offering a vigorous response to the price barriers that limit the availability of scholarship to readers. The scientific community, for some time now, has been publishing research findings through open-access platforms, such as the Public Library of Science (PLoS), to share their work in a timely manner.

In contrast, humanities scholars have been slow to embrace such platforms, but humanities-focused open access now comes in a variety of packages from library consortia, such as the Open Library of the Humanities, to scholarly journals, such as Digital Humanities Quarterly, and independent presses, such as Open Humanities Press and punctum books. All of these open-access formats reflect a simple, yet seemingly radical, ideal: scholarly writing wants readers. This ideal, however, has an uncomfortable by-product: if scholarly work becomes more accessible to readers, the work becomes more vulnerable and its reception becomes more transparent. While there are multiple implications for this vulnerability and transparency within the context of open access, a key one is the relevance of open annotation practices for innovations in peer review.

Social Annotation and Open Peer Review

Reading has almost always been a social act, but I want to suggest that reading hasn’t been this social since the Middle Ages. An important distinction, however, must be made. Whereas now the social nature of reading is enhanced through ubiquity and accessibility, reading during the Middle Ages was social because of scarcity and inaccessibility. Digital texts thrive on speed, scale and access, offering multiple opportunities for encounters with readers. Medieval texts and readers were relatively scarce, raising the value and utility of the single book, which might be used by generations of commentators for interpretations of Aesop’s fables in the classroom to legal glosses on canon law. From these two very different contexts emerge an emphasis on commentary and annotation, which establish a text’s value and use.

Unfortunately, the potential of this social culture of commentary is often squandered, especially within traditional methods of double-blind peer review. I have been persuaded by Kathleen Fitzpatrick and Martin Paul Eve, among others, that open peer review (even in partially open formats) offers more benefits than double-blind peer review for the following reasons.

1. Open review makes commentary more transparent. Open peer review is the equivalent of a Microsoft Word document that tracks changes, showing markup. Many medieval manuscripts and early printed books were produced in anticipation of this marked-up state, with complex textual apparatus, including space for interlineal glosses and marginalia. Within open review formats, the comments of writers and reviewers are made available to all, encouraging vigorous dialogue. For example, the Modern Language Association Commons is currently hosting an open review of the volume Digital Pedagogy in the Humanities, which uses a commentary platform that allows for discussion between reviewers and writers. Such an open format allows writers to evaluate the feedback intelligently, which I have witnessed in two open-review experiments hosted by the journal Postmedieval and Media Commons Press. Writers can assess feedback by asking themselves questions, such as, “Is this just one reviewer’s agenda or is this critique shared by others?” Perhaps most importantly, the transparency of open review reveals bias. If a reviewer has a clear bias, the community of reviewers can help to identify it.
2. Open review enhances the utility and relevance of the commentary. Within open-review platforms, reviewers are often self-selected, based on their investment and expertise, as opposed to responding to a request from an editor to review a manuscript (which may not reflect the reviewer’s interests or expertise).
3. Open review allows for a large number of reviewers. Rather than limit the task of review to a handful of reviewers, work shared in open review is crowdsourced and potentially subject to a large volume of commentary. The work of reviewing could then be distributed, making it less of a burden upon individual reviewers and enriching and enlarging the community invested in the work.
4. Open review treats scholarly work as it really is: work in progress. Finished work is a myth, despite our emphasis on products. An exciting new project, the Open Access Companion to The Canterbury Tales, refers to its first incarnation as a kind of Netflix-like “first season,” recognizing that its value will be maintained or enhanced through accumulation and evolution.
5. Open review maximizes the value, relevance and impact of the work. Years ago, I asked a senior scholar about an argument he made in his first book and was shocked when he replied, “I don’t believe that anymore.” Now that I have published my own work and have revised my thoughts about some aspects of it, I see this as a natural consequence of doing scholarly work. We often change our minds, especially after being exposed to other reasoned critiques of our work. Open review formats could therefore continue postprint, making book reviews more significant and useful. The book review process would become more dynamic -- authors (and other reviewers) could respond to and correct outrageous or uninformed claims in reviews.

Recommendations

We should all be moving toward open-review practices and publicly accessible review platforms, but given the precarious positions of many scholars and publishers, we should proceed with caution. After all, blind forms of review have often allowed work to stand on its own and protected scholars from bias and career-damaging critiques. Keeping in mind Kathleen Fitzpatrick and Avi Santo’s important call for “structured flexibility” in developing protocols and tools for open review, I offer the following recommendations:

1. Make comments publicly available, even within double-blind formats. Whereas anonymity often protects the identities of junior scholars or reviewers who might be given little consideration because of their professional status, I see little benefit from keeping commentary hidden, especially on open-access platforms. While the redaction of comments keeps some critiques out of the hands of tenure review committees, such transparency would allow the larger scholarly community to redress critiques that are useless, unfair or biased. Even within such an open platform, however, editors would need to moderate commentary, especially to prevent trolling, spamming and harassment. If editors want to limit the feedback authors receive, they could open the comments only to reviewers, which could provide a forum for reconciling confusing or contradictory feedback.
2. Adopt single-blind formats, in which the author remains anonymous and the names of reviewers are divulged. This kind of limited open review maintains transparency during the review process, while at the same time mitigating the possible embarrassment or damage to the promotion of a scholar whose work results in public rejection. Many publishers already use a limited form of single-blind review, revealing the name of the author to two or three reviewers only known by the editor. This format protects the reviewers, which can be beneficial for junior or less established scholars who want their reviews to be taken seriously, but it also licenses reviewers to pursue critiques they may not be willing to stand behind. By contrast, an open-review platform that reveals the names of reviewers to the public would encourage responsible critiques and provide valuable contexts, such as a reviewer’s scholarly perspectives or preferences, for feedback that would otherwise be unavailable to authors.
3. Establish multistage processes that combine blind and open review formats. Even within double-blind review, established authors are often identifiable because of their reputations for particular kinds of scholarship or areas of expertise. For such known quantities, even the most transparent forms of open review may be appropriate. New scholars to a field, however, may benefit from multistage processes, in which their work is subject first to blind review before being deemed publishable and then vetted through open review. These hybridized formats would be especially appropriate for well-established journals and presses that already have active and vigorous scholarly communities that are seeking to make their work more available to the public at large.
4. Create spaces for postpublication open review. Many medieval manuscripts survive marked and mediated by the hands of multiple marginal commentators, creating a readerly trail that medievalists follow to track the way the work has been received over time. Today, our book reviews are too often limited to the views of individual scholars, who may not be invested in the work they are reviewing. Postpublication book or article review spaces would open up and crowdsource the reception of the work, providing opportunities for authors to respond to feedback from multiple interested reviewers and make important revisions to their publications.

Open Access Needs Open Review to Be Open

It is important to stress that open access, even in its most liberal forms, does not require open review -- double-blind processes can continue unabated. Open-access publishers who continue to use blind review will not undermine their efforts to make scholarly work accessible. However, those publishers will not fully succeed in making this work open.

The democratic potential and ethic of openness is not fully realized without open review, which would provide opportunities for scholarly dialogue and critique throughout the writing process and beyond. The quality, range and significance of work could be greatly enhanced, offering a distributed network of invested writers and reviewers, rather than small cohorts of experts and exclusive publishing priesthoods.