The Holy Grail of Critical Thinking

Critical thinking has long been the Holy Grail of higher education. We know we want our students to be able to think critically, and we often even know when it happens, but it’s tough to pinpoint exactly what is happening, or why what is happening is happening.

It’s sort of like Justice Potter Stewart’s test for obscenity, “I know it when I see it.”

But because we are now in the Age of Assessment and Accountability™[1] the question: “Is our kids learning?” is omnipresent.

While I am obsessive about assessment at the class level, constantly seeking feedback from students to help me understand their experiences of learning, I am a skeptic of large-scale assessment projects.

The K-12 focus on corporate reform and its standardized education model – You will become college and career ready and fill in the scantron bubbles to prove it! – has been disastrous, leaving students who take this route less prepared for the rigors of higher ed. It also has flattened the K-12 experience, leading to the marginalization of vocational education.

A high school education seems to mean less than ever.

I previously gave a lot of reasons why I was suspicious of the national, Valid Assessment of Learning in Undergraduate Education (VALUE) project, who recently released their initial report based on their first two years of work. 

The twist on the VALUE approach, as opposed to our K-12 assessment regime's reliance on standardized tests, is that they are examining actual student artifacts using a “critical thinking” rubric with five dimensions rated on a 0-4 scale.

I encourage everyone involved in higher ed to read the study. It’s good, it’s clear, and it advances the conversation on critical thinking while also including all appropriate cautions. They present the data without also establishing threshold or cut scores, which we know can be essentially arbitrary. They also warn against using the data to judge the effectiveness of individual instructors.

The most interesting takeaway for me is that students struggle with “drawing conclusions or placing the issue in a meaningful context,” which informs my suspicions about how students view schoolwork as something separate from learning and life. This is an insight that can help instructors think more deeply about pedagogical approaches that will enable students to reach the more sophisticated levels of critical thinking we claim to value.

And yet, as I read the study, I feel almost certain that forces will conspire to make bad use of it.

I wish my mind didn’t go to these places, but I cannot help it.

The temptation for accountability will be so strong that the cautions against establishing thresholds or judging schools and faculty on those cut scores will be swept away, the same way they were swept away in K-12 education.

For example, “Value-added models” for teacher effectiveness in K-12 escaped the lab, going from, “possibly interesting way of providing insight into what’s happening in schools and classrooms” to the “metric to rule them all,” resulting in predictably bad policies and practices being established. 

Some of those who want institutions to be accountable will be well-meaning, seeking to help students and families make the best possible educational choices and have the best possible educational experiences.

But we also have a Secretary of Education who believes college faculty and administrators are ideological enemies of America. “Accountability” focused assessment is just another tool that can be used to bring them to heel.

The most promising part of the study is in its honoring of the complexity of learning and its acknowledgment that critical thinking can show up in many different forms across different types of student artifacts.

But now that the study has been established with its rubrics and areas for improvement, once accountability comes calling, we run the risk of falling into the same “teaching to the test” trap as has happened in K-12 education.

I believe this because I’ve experienced it in teaching writing. At some point in just about every writing teacher’s career, upon witnessing the ways student writing artifacts fall short of what you think a good “essay” should look like, you turn to rubrics and templates as way to guide them to the goal.

The problem, at least as I’ve experienced it, is that the rubric itself begins to dominate the process. Rather than wrestling with the material as writers do in the real world, seeking to tame the problem at hand into a custom piece of writing that engages with audience, purpose, and occasion, I began teaching techniques and “strategies.” For sure, I provided them with more sophisticated and flexible strategies than the five-paragraph essay models they’d arrived with, but ultimately, I realized I was doing little to help them develop the much deeper engagement necessary if they were going to “think” as writers do.

I was helping them produce artifacts that resembled writing – and the critical thinking that requires – without actually being reflective of the kind of process that will one day turn them into flexible, adaptive writers.

In other words, they were creating imitations meant to pass an assessment rubric, rather than the real thing that has to work with a real audience. It really was a college version of the five-paragraph essay, sufficient to temporarily pass a test. Meaningless beyond the classroom.

If I’m going to drive students towards learning critical thinking, I have to ask them to think critically in all the dimensions the concept entails.

For me, critical thinking is this: The ability to conceive, design, and solve a problem of the student’s own making. A good critical thinker both asks the question and provides an answer.

Where most of our tools of assessment fall short – including much of my own past pedagogical practices – is in not giving students sufficient freedom to craft the questions they want to answer. When anxiety about outcomes is highest – and an assessment regime with potentially punitive outcomes is virtually guaranteed to generate anxiety – the natural response is to narrow the questions down to something the students can  “answer.”

But in doing this we fence students off from the greatest pleasures of learning, using those critical thinking skills to navigate a world in which individual agency matters.

I think faculty sometimes get frustrated when students don’t “naturally” take to what we have to offer, wondering what they have against “learning,” but perhaps this is because, for them, so much of school has been organized around “unnatural” things.

Rather than believing that the primary reason students fall short in the VALUE rubric is because they lack critical thinking skills, maybe at least a portion of the blame falls on our assignments.

Maybe they’re not reaching the top of the pyramid because we’re not giving them a compelling reason to climb.

So, while I am fascinated by the findings generated through the use of the VALUE rubric, maybe we should remember the lesson of the Grail Myth, that the holy relic itself is likely to always remain outside our grasp, that it is the quest itself that matters.