Meanings and Metrics

Many self-styled reformers have called on (and called out) colleges and universities to systematically assess how well we educate our students. Margaret Spellings’ Commission on the Future of Higher Education decried our lack of clear, comparative measures of academic success. Some administrators have responded with initiatives like the Collegiate Learning Assessment, designed to quantify and rank the success with which institutions achieve liberal learning outcomes. “Assessment,” in short, has become a word to conjure with.

In the face of this rising sentiment, many humanities faculty respond like King Canute, taking arms against the tide. We humanists are notoriously hostile to systems of assessment. We tend to believe that the most important effects of a humanities education resist measurement: nuanced communication skills, reflective dialogue between theory and interpretation, attention to context and complexity. Conversely the outcomes that can be most readily measured seem to us the least salient: informational content in a sub-discipline, performance of competent analyses according to check-listed rubrics.

Humanists tend also to look askance at the abbreviated time-frame of many assessment tools, whether these tools test student performance at a single moment or mark change in a “formative-summative” sequence. To the contrary (our experience tells us), the most powerful learning in the humanities takes place in ways that are meandering, iterative, self-reflexive, and unpredictable. “We murder to dissect,” Wordsworth famously wrote. The machinery of evaluation, we worry, threatens to kill the soul in the machine.

And so we become assessment Luddites. We sabotage efforts to systematically evaluate how well we are educating students via the time-honored faculty practices of filibustering critique, committee inertia, or sheer disengagement. My own experience in teaching American Studies and leading civic-engagement programs makes me sympathetic to the skepticism. I have rarely seen evaluative tools that do justice to my experience or that of my students. And yet, I want to argue, it is time for humanists to move beyond Luddism and constructively engage the advocates for student assessment.

There are two overriding reasons: one strategic, the other educational.

First, the current calls for assessment are part of a larger crisis of legitimacy in U.S. higher education—a crisis that faculty ignore at our peril. The crisis has many causes: tuition increases that have long outstripped the growth in the cost of living, the erosion of educational access and attainment, culture wars over perceived political bias in the academy. In the context of these discontents, calls for “accountability” have sometimes masked efforts to police campus politics; and too often they have oriented higher education toward instrumental goals of job training and economic competitiveness.

Yet (especially in a time of scarcity and crisis) it is a fair challenge to the academy that we be accountable for the vast resources and autonomy to which we lay claim—that we offer a compelling argument about our value to the larger society. Precisely because others have their own reductionist agendas of how to measure success in higher education, we need to offer our own vision of means and ends. The most self-damaging response we can make is to build a defensive bulwark of guild privileges around ourselves.

More substantively, it is not simply in our interest but in the best traditions of the humanities to pose the questions that underlie the calls for assessment. What constitutes a good liberal education, one that is emancipatory and transformative for students? What is the distinctive role of the humanities in that education? How do we know whether our educational practices embody these values? It is hard to find assessment tools that advance rich answers to these questions; all the more reason for skeptical humanists to enter the conversation.

How, then, should we think about assessing undergraduate learning in the humanities? Let me begin (like the cultural historian I am) by unpacking a story: the story of a student whose college experience and humanities learning I know a bit about. In the early 1980s he attended a selective liberal-arts institution, far from the state and community where he attended high school. He was (or so he recalls) intellectually curious, academically successful, and intensely concerned with matters of racial identity and politics; for he was a non-white student in a largely white institution, active in the anti-apartheid movement so central to campus politics at that moment. Humanities learning seems to have played a crucial role in all this: he recounts dormitory discussions about Franz Fanon and complex arguments about whether Heart of Darkness was racist. And yet despite all this mobilization of heart and mind and action, he recalls never feeling fully engaged by his college experience, never fully himself, never fully at home. He decided after two years to transfer to an urban university, from which he eventually graduated.

How do we assess this student’s learning? I mean this in both senses: how successful was his experience, and what tools would we use to measure that success? Should the downward blip on his college’s retention rate, or the low marks that he might have given on a NSSE-style survey, be assessed as tokens of failure? Should his good grades, successful transfer, and timely graduation lead to a more positive view?

Moreover, how do we evaluate the role of the humanities in his education? In retrospect, he describes literary and philosophical texts as testing-grounds for his ambivalence; it was through the production of readings that he worked out his attitude toward the liberal-arts college and himself. Should we take his stories of identity crisis, fear, and self-masking as evidence of the failure of the humanities to nurture and catalyze his development? Or do his textured memories of reading Conrad and Fanon -- and the powerful autobiography in which he recounts these experiences fifteen years later -- point to the efficacy with which a humanities education taught him critical thinking, civic engagement, and communication skills?

This may be an unfair example with which to frame the problem. On the one hand, we know far too little about this student’s academic experience to measure its effectiveness. On the other hand, we know too much. For the student is of course Barack Obama, recounting the story of his time at Occidental College in his 1995 memoir, Dreams from My Father. Now an assessment advocate might justly complain that this is the kind of anecdotal cherry-picking with which intransigent humanists typically sabotage the debate. Yet I’d argue that Obama’s story is good to think with, when we think about student learning in the humanities. It points to what inadequate assessment tools tend to miss, and it points to some qualities that we would want adequate ones to capture. Any evaluation system that cannot make sense of the success of Obama’s education -- and the role of the humanities in it -- will not pass the smell test for skeptical humanists.

What can we glean from the story? First, that learning in the humanities is developmental but not linear. It does not unfold as a vector from ignorance to knowledge or a skill-building sequence toward proficiency. The memoir recounts an experience that is cumulative and transformative, but one that moves in bursts and stumbles, trials and errors. The young Obama grapples with whether Heart of Darkness colludes with the racist colonialism that is its theme; an activist friend criticizes him for taking it seriously; he defends his interest to yet a third friend, someone who plays a catalytic role in his Occidental career.

Secondly, this developmental rhythm, at once exploratory and recursive, tell us something about the “geography” of effective learning in the humanities: it is not bounded by the curriculum. We never learn whether Obama was assigned Heart of Darkness in a course. Yet it is clear that if his engagement with the novella began in the classroom, it did not end there. His response to it is forged in political argument and friendly banter. Indeed it is the contradictory, promiscuous impact of Conrad’s story, wandering across his studies, his identity crisis, and his activism, that gives it a place in his memoir. The novella is important precisely because it cannot be sorted neatly under “curricular” or “non-curricular” purposes, because it worked at cross-purposes, forcing him to double back on its mixed meanings, enabling him to work through his own self-division. Dreams from My Father maps a process in which texts, classes, friends, political struggles, and personal turmoil all work together to nurture the author’s development.

We can discern here some of “learning outcomes” that we would want assessment tools to register. Humanities learning is iterative, cumulative, integrative, and open-ended -- but, then again, so is the process of learning tennis. What is distinctive to the humanities, I would argue, is that these qualities nurture the student’s capacity for meaning-making and reflection; and that meaning-making and reflection, in turn, nurture the student’s capacity for self-making and engagement (ethical, civic, vocational) in the world.

Such outcomes require the acquisition of knowledge and skills, to be sure, and a good education in the humanities will equip students with the capacity to (say) interpret, contextualize, and discuss Heart of Darkness. Yet it will also immerse students in relationships and projects (curricular as well as co-curricular, personal as well as civic) that activate their knowledge and skills. Knowing how to craft a reading, analyze an argument, or place an idea in social context is important -- but only if (at the same time, seamlessly) it enables students to act mindfully in the world and to reflect on their actions in dialogue with others. Student learning in the humanities is inseparable from student learning with the humanities.

This helps to explain why humanists are allergic to evaluative instruments that test cultural knowledge or interpretive literacy through the time-compressed performance of some sort of demonstration task. This is like assessing a couple’s relationship by asking them to talk about some assigned topic and measuring the intimacy, honesty, and empathy they display. The capacity for intimate, honest, empathetic conversation is of course essential to relationships, but successful couples use it in diverse ways, at varying frequencies, to shifting purposes. It cannot be helpfully reduced to a model or ranked on a metric. In the same way, mastery of cultural knowledge and interpretive skill is an important means of humanities learning, but not a proxy for it.

What, then, would a robust assessment practice look like? It would embody the qualities that typify humanities learning itself. It would be iterative: gathering and evaluating portfolios of material from the whole arc of the student’s career. It would be exploratory and integrative: asking students to include in those portfolios materials in which they are not only learning about the humanities in their course of study, but also using it in their civic, ethical, vocational, and personal development.

It would be autobiographical: requiring students to narrate and thematize that development, to frame their portfolios with their own, small versions of Obama’s memoir. And it would be reflective: calling on them at threshold-moments to plan and take stock, to evaluate their successes and failures, and (equally important) to make explicit what they count as success and failure in their education. This last point is crucial: humanities assessment (like humanities learning) is intrinsically dialogical and open-ended. Indeed the sine qua non of a successful humanities education may be precisely that it equips students to discuss and contest the question, “Has my education been a success?” with their teachers and their peers.

Fortunately, we have good models, scattered across the academy, for the kind of assessment system I am advocating. Wabash College, Portland State University, and LaGuardia Community College—among many other institutions -- have pioneered the use of electronic portfolios for the assembling and assessing of student work. Bard, Hampshire, Marlboro, and other “alternative” liberal arts institutions choreograph their curricula around threshold-moments in which students take stock of, plan, and set goals for their course of study in dialogue with faculty supervisors. The Association of American Colleges and Universities is working with faculty across the United States to develop frameworks for the systematic use of portfolios in both “milestone” and “culminating” assessments.

I am mindful that the model I am sketching is bound to give the assessment reformers heartburn. Portfolios framed (like the pages of the Talmud) by autobiographies, reflection statements, and contestatory dialogue; student work assembled in narratives of meaning-making, rather than being measured as evidence of mastery -- this is surely not what the Spellings Commission meant when it called on academics to take assessment seriously. For the reformers want an efficient, transparent, portable metric of effective teaching and learning: a tool that can quantify the value-added of a college education, of skills learned and knowledge deployed, in comparative rankings.

For skeptical humanists, however, such a tool is like an intellectual microwave: you can heat something up with it and assess the taste, but you will not glean very much about the nutritional value of learning that must be simmered and seasoned. Our insistence on a “slow food” model of evaluation is not obstructionist; it is what humanists mean by taking assessment seriously.

Speaking for myself, I am grateful that the reformers have pushed us to articulate the goals and values of a humanities education and the practices that would adequately evaluate our successes and failures. I’m willing to work on assessment under the banner of NO LUDDITES -- but only so long as the other side of the banner reads NO SHORT CUTS. Sometimes, as Leopold Bloom muses in Ulysses, “The longest way round is the shortest way home.” I suspect that the author of Dreams from My Father would agree.