External Peer Review: Why Not for Teaching?

Imagine you have completed a scholarly article, book or creative product that you intend as a contribution to your discipline. Who will evaluate your work, attest to its quality and determine whether it is published or exhibited? Who will review the work when you are up for tenure and promotion or contract renewal?

Now, in your mind’s eye, imagine a person who is likely to review the quality of your teaching for professional benchmarks.

I wager that you can put a name and highly familiar face to that second scenario. Colleagues in our departments and programs, whether department chairs, assigned mentors or members of a teaching committee, almost always conduct peer reviews of teaching. Frequently enough, we are responsible for inviting a colleague of our choice to review some course materials, visit class and craft a letter based on their observations. When it comes to our scholarship, however, peers external not only to our departments but also to our colleges and universities conduct reviews behind a double blind of anonymity.

Why? Because the external review of scholarship enlists disciplinary expertise; the scholarly excellence, contribution and innovation of a Victorianist’s work is well judged by another Victorianist, not an 18th-century Americanist. The validity and scientific significance of a microbiologist’s work may not be best judged by an ecologist.

An equally important justification for seeking the external review is the aspiration to objectivity. The external reviewer is farther from home and presumably unbiased or personally affected by the outcome of the review and its consequences for the scholar’s career. The reviewer feels less social pressure to overpraise work by a beloved colleague or to eviscerate the work of one who doesn’t pull their weight in department service or who gets on the nerves.

But while external peer reviews of scholarship or creative work give the benefit of both expertise and objectivity, peer reviews of teaching often suffer from the lack of both.

If you have ever served on a tenure and promotion committee or in an academic leadership position, you have probably read teaching letters that vary widely in both quality and helpfulness. I myself am eager to get a window into the classroom through apertures provided by the professor’s portfolio and from the peer reviewer. I feel much more confident recommending candidates when I can see alignment between not only the professor’s expressed intentions as a teacher and the students’ evaluations but also the peer reviewer’s report. I know I am not alone in searching for the third leg of that three-legged stool, and I am sometimes disappointed to be able to see very little through a veil of laudatory and vague statements that lack observational evidence.

Lee Schulman, past president of the Carnegie Foundation for the Advancement of Teaching, has argued that research is “community property” but teaching is done in isolation: “We close the classroom door and experience pedagogical solitude, whereas in our life as scholars, we are members of active communities: communities of conversation, communities of evaluation …” He proposes that we must also make teaching community property if we will elevate its value and create conditions for its improvement and excellence. To make teaching community property, he argues that it must be: 1) advanced within the disciplines, 2) demonstrated through reviewable products (syllabi, lesson plans, assignments) and 3) reviewed by peers. The structure for this last recommendation is worth defining, as current peer-review processes are frequently absent both objectivity and expertise.

Ensuring Objectivity

As in peer review of scholarship, the peer reviewer of teaching should be external to the department and ideally unknown to the professor being observed. The following proposals, while alien to higher ed culture, are necessary if we are to protect the integrity of peer review of teaching from the insularity of individual departments and even campuses, from the subjectivity of colleagues who have social concerns as well as skin in the game. I offer three options, in descending order of plausibility, of the people who could conduct peer reviews of teaching:

Colleagues within the institution but outside the candidate’s home department. For example, college or university-based interdisciplinary teaching committees could conduct such reviews. Two or three committee members could serve as reviewers, providing the benefit of multiple perspectives. Disciplinary familiarity rather than expertise is sufficient. Faculty do not need to be checked for communicating accurate content to undergraduates; the reviews should focus less on content and more on teaching practice. This option is more conveniently “in house,” although at least not from the exact same room in that house.

Academics in a consortium of peer institutions. In areas dense with higher ed institutions, colleagues could visit peer campuses to conduct reviews. For more remote institutions, partnering colleagues might review course materials and videotaped class sessions. The consortiums could be organized by institutional sector: doctoral research, selective liberal arts, highly accessible liberal arts, Catholic schools, HBCUs, community colleges and so on. Once the institutions established the agreement, a department chair could reach out to any other chair within the consortium to seek a peer reviewer of teaching. Each member institution would commit to conducting and receiving reviews within the exchange. Besides addressing a fundamental problem with how we evaluate and recognize good teaching, such exchanges would be professionally enriching opportunities to observe and learn from colleagues in other institutions. I would love to see foundations support these types of collaborations for evaluating and enhancing the quality of teaching across institutions.

Consultants with expertise in best teaching practices and how people learn. They might be higher ed teaching-award winners, education professors or faculty development/teaching and learning professionals. This last option is the least likely to get traction because it is most challenging to higher ed culture, would cost money and enlists reviewers whom some might not recognize as peer reviewers in the strictest sense. But if we value teaching enough to submit it to the judgment of people who are both objective and expert reviewers, as we do with our research, this beautifully meets the goal.

Ensuring Expertise

Peer review of teaching should include evaluation of course materials and course assessments in addition to classroom teaching. No matter who conducts the review, it must be standards based, not idiosyncratic. We actually know quite a lot about what distinguishes effective teaching at this point in the history of higher ed, but it is the rare department or university that has articulated specific standards for quality teaching, and even rarer to find a rubric or instrument that directs the reviewers’ attention.

I once received a peer review of teaching that devoted a few precious sentences to my not having a policy against eating in class. There is nothing in many departmental peer-review processes to prevent such myopia. When I review manuscripts for a scholarly journal, and before I write a narrative response to the work to justify my decision or give helpful feedback to the author, I answer a set of questions about the originality of argument, clarity of writing, soundness of methodology and significance of findings.

Any system of peer review requires, first, an articulation of standards for effective teaching. Based on the scholarship of teaching and learning, the standards could include items for clarity, organization, student engagement, teacher-student rapport and assessment practices. Individual instructors will have distinct ways of enacting standards, so items should not be too prescriptive. For example, if the standard is “instructor clarified abstract or challenging concepts,” a faculty member might get high marks for actions as various as including in-time class assessment techniques to gauge comprehension before moving on to a new topic, including helpfully concrete analogies or conducting a clarifying demonstration of a principle.

An agreed-upon set of standards is ideally communicated in a rubric or checklist. The most important thing is to communicate the standards and direct the reviewer’s attention appropriately.

No one defining these standards has to start from scratch. An excellent resource is Nancy Van Note Chism’s Peer Review of Teaching: A Sourcebook. Years ago, I led an initiative out of a Teaching and Learning Center that invited departmental teams to work with us to articulate standards and create instruments with the help of this resource book, which provides models in a variety of formats (checklists, rubrics, narrative forms) and a series of well-articulated standards statements. We encouraged the departmental teams to bring their standards and instruments back to the department for discussion and eventual adoption.

The system should allow for "norming" among the various reviewers using the same standards and instrument. Just as reviewers for SAT essays, or TAs grading for one class need to align their expectations to achieve consistency, so should peer reviewers of teaching.

I have focused on peer review, but it is worth saying that these reviews should be part of a portfolio that contains evidence from three sources. In The Academic Portfolio, Peter Seldin recommends evidence from students, peers and the teachers themselves in a reflective portfolio, including a teaching statement that frames the assembled evidence.

If we value teaching, we should evaluate it with both objectivity and expertise and reward those who demonstrate excellence. Our current systems are substandard. But we have the knowledge we need to create better processes. Now we need the will and courage to do things differently for the benefit of our students.