You have /5 articles left.
Sign up for a free account or log in.
As Walt Whitman said, “Do I repeat myself? Very well then I repeat myself.”
I’ve said this a couple times before, but let me say it again:
1. Algorithmic grading of student writing is an abomination. It enshrines terrible, counterproductive behaviors which have exactly zero relationship to the kinds of experiences that are necessary for writers to develop their writing practices -- the skills, knowledge, attitudes, and habits of mind that writers employ as they’re writing.
2. The use of algorithmic grading is an attack on the value of the academic labor of both teachers and students. Their use signals that this work simply does not matter.
3. Plus, no matter what anyone tells you, algorithmic grading really doesn’t work for shit.
Deep breath…
I would like to stop repeating myself, but a segment from Saturday’s episode of NPR’s Weekend Edition informs me that “more states are opting to robo-grade student essays by computer.”
Head meet desk…deep breaths…
I’m going to bright side this and choose to believe that at least these stories bring the issue into the light, so we can push back against the adoption of these policies, but I’m starting to wonder if maybe the megaphones of those of us who are pushing back aren’t loud enough to drown out the forces which seem to be pushing this technology into wider use despite it BEING AN ABOMINATION!
Deep breaths…in…out…in…out…
The NPR story follows what is now a familiar trope. Those who develop and provide the software admit that while the old stuff was maybe not so great, the new stuff is really working like gangbusters: believe us. In this case, we hear from a vice president for research at Pearson and a senior research scientist at Education Testing Service (ETS). I can’t imagine them having any self-interest when it comes to discussing the use and efficacy of algorithmic grading, can you?
We also get the de rigueur appearance of some actual teachers who always say something like, “this idea is bananas” because of course it is totally bananas. The teachers seem to get less time in these articles because what is there to say other than “this is bananas”?
Les Perelman of MIT does his usual yeoman’s work of demonstrating how the algorithms are easily fooled by his Babel (Basic Automatic B.S. Essay Language) Generator,[1] after which the NPR correspondent chuckles heartily, goes back to the folks from Pearson and ETS and says, “You’re kidding with this shit, right? I just saw Les Perelman’s algorithm fool your algorithm with utter nonsense.”
No, that does not happen because NPR has a nearly slavish devotion to “balance” even if one side of the debate is talking clear common sense, while the other is peddling something that is bananas.
A GRE tutor tells NPR that in order to get a good score he explicitly trains students in “fabricating evidence and fabricating fake studies.”
The tutor provides a MadLibs-style template to help students ace the GRE essay scoring algorithm.
“A [pick any year] study by Professor [fill in any old name] at the [insert your favorite university] in which the authors analyze [summarize the crux of the debate here], researchers discovered that [insert compelling data here] ... and that [offer more invented, persuasive evidence here.] This demonstrates that [go to town boosting your thesis here!]”[2]
Confronted with this evidence, Madnani of ETS responds that they “see a lot of that,” but “it’s not the end of the world.”
He continues, “But if the goal of the assessment is to test whether you are a good English writer, then the facts are secondary.”
Here it is folks, we’re all the way through the looking glass. The facts are secondary.
In case it needs saying, you are not a good writer if you’re making stuff up, unless the point of the writing is to make stuff up. Writing is thinking. Making stuff up and pretending like it’s true is not the kind of thinking good writing requires.
Writing is thinking. Writing is thinking. Writing is thinking. Writing is thinking. Writing is thinking.
Writing is thinking. These assessments require students to engage in thinking that is antithetical to writing well.
For students to learn to write well they need to do the kind of thinking writers do. One very obvious aspect of that thinking is trying writing to specific audiences and conveying information and ideas that are both accurate and true.
Facts cannot be secondary because in writing, nothing is secondary. Writing is thinking, writing is thinking, writing is thinking where many things are happening simultaneously as the writer builds their message, word by word, paragraph by paragraph, idea by idea. Writing utilizing fake information is not practice for any kind of real writing.
Perhaps I should be excited, since I can think of no better argument for the necessity of the existence of my forthcoming book Why They Can’t Write: Killing the Five-Paragraph Essay and Other Necessities than the increased adoption of robo-grading, but it honestly breaks my heart to consider the damage we’re doing to students.
It almost boggles my mind that my book is not only necessary, but seems increasingly urgent.
By the way, Walt Whitman never said, “Do I repeat myself? Very well then I repeat myself.” The algorithmic grader would never have caught such an error. It wouldn’t have caught it if I said it was from Adolf Hitler or SpongeBob SquarePants, either. It literally wouldn’t matter.
The real quote from Whitman’s “Song of Myself” is:
Do I contradict myself?
Very well then I contradict myself,
(I am large, I contain multitudes.)
We all contain multitudes. Why would we require our young people to deny these multitudes in order to please these algorithms?
I mean, I know why – it’s about cost, efficiency, and profit - but I’m asking those who believe in these things into confronting this question for themselves. Nitin Madnani of ETS has convinced himself that facts are secondary. What else are we saying is secondary when these initiatives are adopted with so little consideration of student development and well-being?
Here’s what I’m saying: it’s bananas.
[1] This is the paragraph Perelman’s algorithm generated. It got a perfect score on the GRE’s algorithm: “History by mimic has not, and presumably never will be precipitously but blithely ensconced. Society will always encompass imaginativeness; many of scrutinizations but a few for an amanuensis. The perjured imaginativeness lies in the area of theory of knowledge but also the field of literature. Instead of enthralling the analysis, grounds constitutes both a disparaging quip and a diligent explanation.”
[2] As the tutor says, it’s kind of fun to fill these out for yourself. Here’s mine: A 2047 study by Professor Mick Jagger at Rolling Stones University in which the authors analyze the physical repercussions of rock and roll music, researchers discovered that the guitar solo in “Sympathy for the Devil” will melt your face, and that repeated exposure will result in additional face melting. This reveals that exposure to rock music is bad for your physical health because people who have had their faces melt off are prone to infection.
