Do Faculty Need an Automated Grading Tool?

Do faculty need an automated tool to help them in grading online class participation?

The answers are no, hell no, and get the heck out of here with that ish.

As reported by Inside Higher Ed, Blackboard is going to introduce a feature that will “recommend” a grade on discussion forum posts based on a combination of algorithms, including the Flesch-Kincaid index (which measures complexity of vocabulary), the type-token ratio (which looks at word variety) and something called the “critical-thinking coefficient.”

The critical thinking coefficient “classifies words according to the degree of critical thinking represented.” I will be honest. I have no idea what that means or how it could be done. I cannot grok how individual words without context and meaning can be classified to represent more or less critical thinking.

Like all of our automated tools Blackboard claims they do not intend to replace human instruction with robots. Analytics chief at the company, John Whitmer, says the tool “will relieve instructors of many tasks of evaluating the quantity of participation, so that they can focus their assessment on the deeper value and meaning in student work.”

The original article collects various opinions on the tool, ranging from cautious okayness to “who are you kidding with this stuff?” Count me as aligned with Jesse Stommel[1] who notes this kind of algorithmic response incentivizes “rote behaviors” in the service of pleasing “arbitrary markers.”

Even though I’m confident it doesn’t, let’s imagine that this stuff actually “works,” in that it provides an accurate reflection of the grade an instructor would assign independent of the algorithm. Is this a good idea under those circumstances?

Still, no. This kind of technology, while meant to spur increased engagement, will almost certainly prove alienating. Now, I’m of the belief that any grades are a hindrance to learning for all the reasons Susan D. Blum articulates in her essay on "ungrading."  But even if you don’t go that far, algorithmic grading reinforces a message incompatible with education.

When it comes to the systems of school and the role of grades, students are more than savvy. In the words of Prof. Blum, they are “taught to focus more on schooling, rather than learning.” If they are in a class where some assignments are human graded, some are algorithmically graded, and others are ungraded, you are essentially signaling some assignments mean more than others, while other assignments mean nothing at all.

I’d like to think we don’t assign meaningless work, but when the course is organized around grades, and assignments are graded by different mechanisms, what other message will students receive?

It’s a bad idea not just because those algorithms don’t divulge anything meaningful about the texts, but because students will quickly learn to game the system, eroding whatever benefit the original discussion exercise sought. Using the algorithm says, “this is not important enough for the teacher to grade, therefore it isn’t important.”

It is my belief that anything that increases the importance of the grade in fact reduces the rigor of the course because it reduces the amount and type of agency students must exercise while engaging with the material. Figuring out how to please an instructor may be difficult, but it isn’t challenging in the fullest sense of the word. Figuring out how to satisfy an algorithm is even less inspiring.

The impetus towards these tools is to lighten the workload for instructors. Many of us have found ourselves overwhelmed with student work, unable to find sufficient time to respond to everything in the way we would wish. Isn’t an algorithmic aid better than nothing?

From a labor/solidarity perspective, the answer is no. When we allow algorithmic substitution for human labor, we are complicit in our own ultimate obviation. There’s plenty of good pedagogical reasons to refuse these algorithms, but we should also not allow our jobs to be degraded to the point where we can be replaced by something that doesn’t really work, but may pass muster for people who don’t care about teaching and learning.

Not to go all Norma Rae, but…Resist!

Based on my own experience, if you have too much to read and grade, my suggestion is to stop grading every last thing. And if you can manage it, go all the way and don’t grade anything.

“But how will the students learn?” some will ask.

Better, I say.

In my writing classes, particularly early in the semester I often assign work that is not only ungraded[2] but on which I don’t even provide formal feedback. For example, in an introductory creative writing class, I will give students a weekend to write a short story under some specific constraint, like a quest narrative, or a story with an animal as a protagonist. We do four of these total and while I read them – very quickly – I respond to none of them in writing.[3]

Instead, we treat them as learning experiences on which to reflect. In class, students exchange copies and read each other’s work and have a short conversation afterward. We then have a full class discussion about triumphs and difficulties associated with the doing. I ask them what their favorite parts of their own work are and what they’d be doing differently if/when they go back to revise.

I am asking them a big question embedded in a bunch of little questions, “What are you learning?” and they are answering it for themselves, rather than waiting for me to tell them.

In other writing classes I assign work with a public facing component, such as leaving a comment on a New York Times op-ed. Credit is achieved by doing it and sending me a link.

This sort of work is particularly important early on in a writing class because I want students to move away from the idea that they may be writing to please me, the teacher, as quickly as possible. I am not their audience.

I also am not responsible for their learning, they are. By removing grades from the equation, they are forced to take ownership of their work. I’ve found this to significantly increase the amount and quality of work students do in my classes. They are driven not to please an arbitrary standard, but to follow their own curiosities.

When I do involve myself by offering much more copious and detailed feedback on their work, the conversations are now different. Rather than debating my judgment on their grade, we are having a discussion about how to fulfill the objectives they’re pursuing out of their own impetus. Laura Gibbs, who teaches entirely online at the University of Oklahoma, has collected some very interesting student comments regarding her ungraded classes which reflect this ethos.

Like Laura Gibbs, I have not found it necessary to drive students toward learning like a team of mules pulling a wagon up a mountain, me cracking the whip next to their heads lest they feel my lash. Give students something worth doing and they’ll do it. Not all of them, but even for those who don’t, you’re doing them an additional service. Rather than being able to offload the responsibility for their non-engagement or failure onto an authority figure, they must confront their own shortcomings.

Another benefit of this approach is it requires me to be significantly more conscious and careful about the work I give students. I have to make sure what I ask them to do is worth doing, even if I were to disappear entirely from the equation.[4]  

Build the semester around experiences rather than assignments, and make sure to fold reflection into those experiences and I swear students will learn.

That is the goal, isn’t it?