A Collaborative Approach to Open Peer Review

Those in the humanities often champion collaboration and the open exchange of ideas, but you wouldn't necessarily know that when you look at the venues they use to share their work. Hybrid Pedagogy seeks to challenge not only how humanists teach, but also how they publish. The six-year-old online journal pursues humanistic values by embracing an editorial process well-established in the sciences: open peer review.

Hybrid Pedagogy’s editorial process is uncommonly inclusive. Whereas many academic journals prize selectivity, Hybrid Pedagogy accepts the vast majority of submissions—about 70 percent, according to the current editor—with the expectation that authors and reviewers work hand-in-glove to revise essays. The resulting articles are short (by academic standards), visually engaging, widely circulated, and more personal and political than those in traditional academic publications.

While print journals embrace open peer review—as is the case with STEM journals such as Atmospheric Chemistry & Physics and PeerJ—the pairing of open peer review with web technology enables new editorial approaches. In a Views article for Inside Higher Ed, Alex Mueller, associate professor of English at the University of Massachusetts in Boston, wrote that combined with open access, open peer review can support new forms of scholarly inquiry.

Such methods have long proliferated in the sciences. For years, physicists have used arxiv.org, the physics pre-print repository, to perform pre-publication review, said Cheryl Ball, editor of the web-text journal Kairos: A Journal of Rhetoric, Technology, and Pedagogy and an associate professor at West Virginia University.

Ball wishes the humanities would move toward an open peer-review model, too. “I think that a lot of disciplines in the humanities are still stuck in that proprietary single-author non-collaborative model, where scholars are afraid to showcase their research before someone senior has put their stamp of approval on it through the traditional peer review process,” she said.

Given that humanities tend to be more open to collaboration in the classroom, perhaps it isn’t surprising that pedagogy scholars have been quicker to embrace collaborative review. By creating an open peer review process that, in many ways, extends the work of publications such as Kairos and Hybrid Pedagogy, like a good pedagogue, practices what it preaches. The project’s editorial process is integral to its pedagogy, but so, too, is its commitment to critical digital pedagogy, which places Brazilian educator Paulo Freire’s “problem-posing education” in the context of the internet.

Kathleen Fitzpatrick, who used MediaCommons Press to open up the editorial process for her Planned Obsolescence: Publishing, Technology, and the Future of the Academy, offered her own account of open peer review.

“It’s a very broad category, and we need many different kinds of projects in order to help it develop,” said Fitzpatrick, now associate executive director and director of scholarly communication at the Modern Language Association. “Those projects might include otherwise conventional review processes that make their readers’ reports [and readers’ identities] available alongside the article, or community-oriented processes that bring together a group of scholars for a discussion of work-in-progress, or processes that reach out to and involve much broader publics.”

Double-Open Review

Open peer review is no panacea. For example, when the journal Nature experimented with open peer review in 2006, only about five percent of authors opted in (although it reported last year that more science journals now are open to the idea). However, electronic platforms have enabled enterprising rhetoric-composition specialists and digital humanists to reshape editorial practices.  

Kairos has conducted open peer review since it launched in 1996. Submissions are distributed to an editorial board via Slack, and as many as 15 reviewers volunteer to discuss a submission. Similarly, The Journal of Interactive Technology and Pedagogy is staffed by an interdisciplinary Editorial Collective whose category editors quickly approve or reject submissions; readers can contribute to post-publication peer review.

And, while it isn’t a peer review per se, Digital Humanities Now uses algorithms and editors-at-large to aggregate and curate gray literature, work that might not otherwise appear in scholarly venues.

Hybrid Pedagogy takes a slightly different approach, using what editor Chris Friend describes as a “double-open peer review.” First, the editor may be familiar with the author, and he knows the reviewers and chooses reviewers based upon the needs of the piece. Second, the authors and reviewers engage in direct conversation. Readers can also contribute post-publication via the Hypothes.is plugin.

Critics have charged that the site’s authors and reviewers are too cozy. It’s a fair critique, .especially from the vantage of a traditional academic journal. However, Hybrid Pedagogy isn’t traditional. Sean Michael Morris, who previously edited the journal, said he accepted “99 percent” of submissions.

“We want to recognize that what’s coming in is a draft, and we’re going to work with these folks until they feel comfortable with what’s being published,” added Morris, an instructional designer at Middlebury College.

Collaboration is integral to Hybrid Pedagogy’s editorial process. Instead of a call for papers, editors circulate “calls for participation.” Whereas a traditional peer review might culminate with an author receiving summative end comments, Hybrid Pedagogy expects authors and reviewers to work through changes using Google Docs’ threaded marginal comments. The site also identifies reviewers and photographers in bylines.

“It gets to a point where our editors are personally invested in these articles and that they’re proud when this work comes out,” said Friend, assistant professor of English at Saint Leo University.

Articles more closely resemble extended blog posts than journal articles. They’re short (around 2,000 words), freely available (Creative Commons license 3.0), more likely to be coauthored, and more likely to be widely read (the site averages 10,000 to 15,000 page views per month).

Institutional Challenges

The project’s much-vaunted hybridity poses logistical challenges. While authors often include their essays in portfolios, editors admit that those pieces are rarely cited in tenure review. The journal is registered as a peer-reviewed journal with the Library of Congress, which means that Hybrid Pedagogy articles appear in Google Scholar; however, many universities are reluctant to regard those publications as scholarship.

Jesse Stommel is co-founder and executive director of the Digital Pedagogy Lab and director of Hybrid. This disconnect affected Stommel, who explained his decision to leave University of Wisconsin-Madison: “While I was initially assured that digital work for broad public audiences would count, I was later told I should wait to do that work until after tenure and focus on traditionally peer-reviewed publications for academic audiences.”

Stommel is now executive director of the Division of Teaching and Learning Technologies at the University of Mary Washington.

Hybrid Pedagogy’s lack of institutional affiliation may contribute to skepticism. While editors enjoy creative liberty, they also rely upon partnerships for funding. Early in the project’s development, a corporate sponsorship created some controversy. (Leadership has since formalized its approach to sponsorship with a Call for Sponsors.)

Hybrid Pedagogy is registered as a nonprofit. The platform also features podcasts, MOOCs and the Digital Pedagogy Lab (DPL), whose public outreach institutes have greatly expanded in scope, frequency and geographic diversity. Morris, who now serves as DPL director, estimated that 175 participants will attend this summer’s institutes, more than twice that of the first institute in 2015.

Perhaps the greatest impediment to academic recognition, however, is that the journal veers well outside the bounds of traditional academic writing. Stommel credits Morris with challenging the publication to “up the ante on the willingness of our writers, including me, to be brave, to really put themselves on a limb, and to take risks.” Morris now serves as director of the Digital Pedagogy Lab.

Well before he became editor, Friend wrote an intimate account of an incident when he was a high school teacher and his students vandalized his classroom because they believed he was gay. The essay isn’t just an exploration of that incident, but also a disarmingly honest assessment of how he believed he had mishandled the situation.

A Different Model

Hybrid Pedagogy fills a unique niche. “It’s a hybrid between the scholarly and the public, between blog posts and manifestos and articles,” Ball said. “It’s meant to be this mixture between external researchers trying to reach a wider public audience and the editors who have taken on a responsibility to synthesize the key topics of the field.”

“It seems to me that Hybrid Pedagogy represents a digital version of longstanding activist-inclined journals such as Radical Teacher,” added Stephen Brookfield, author of Becoming a Critically Reflective Teacher. “Something like Hybrid Pedagogy is vital because it creates a support network where people can share experiences and get feedback on their experimentations with radical practice.”

Brookfield is the John Ireland Endowed Chair in the College of Education, Leadership and Counseling at the University of St. Thomas.

“Critical Digital Pedagogy must also be a method of resistance and humanization,” Stommel wrote in 2014. “It is not simply work done in the mind, on paper or on screen; it is work that must be done on the ground.” 

Will Fenton is a teaching associate at Fordham University, fellow at MLA Connected Academics and the Library Company of Philadelphia, and educational technology columnist at PC Magazine.