You have /5 articles left.
Sign up for a free account or log in.
Digital alternatives to traditional end-of-semester student evaluations seem more numerous by the day. One new tool hopes to advance that landscape with the help of artificial intelligence.
Hubert, launched last fall and currently in use by more than 600 instructors worldwide, appears to students as a chat bot that asks questions about the quality of the class and the teaching. The conversational messenger format is designed to make students feel more comfortable sharing honest praise and criticism, and the low amount of required effort allows instructors to collect feedback at several points throughout the semester.
The instructor side is where Hubert most differs from similar products. Hubert organizes and synthesizes feedback into a report viewable on an online dashboard. Strengths and areas of improvement appear in separate columns, collated by an AI analysis of repeatedly invoked phrases and sentiments.
“The instructor gets a fast and comprehensive overview of what the class thinks and can choose to dig down really deep to find the reason,” said Viktor Nordmark, chief marketing officer at Hubert. “Questions such as ‘What did my students think of the lab sessions?’ can now be answered in a few seconds.”
The beta version was released seven months ago, after a six-month development period, according to Nordmark. Instructors and students can use the tool for free; it cost the Swedish start-up with four founders and two other employees $600,000 to build, and maintenance costs are $200 per month. The company currently isn't generating revenue but hopes to create paid partnerships with employers and higher education administrative offices.
The name “Hubert” is a play on the word “hub,” because the tool is designed to serve as a hub of feedback, Nordmark said.
What It Can (and Can’t) Do
Hubert asks students variations on four basic questions: What could the instructor do to improve the course? Is there anything the instructor should stop doing? What's working well? How has the class been over all?
It is persistent -- if you respond that nothing about the course should be changed, Hubert follows up with “Surely there must be something that could be improved?”
Instructors send links to Hubert via email or through the learning management system. Nordmark said the company will build integrations with all the major LMSes in the near future. The company doesn’t sell email addresses it collects through the Hubert platform, Nordmark said.
As of early this semester, professors have started soliciting feedback through Hubert that will shape their approach to teaching the rest. Initial reviews of the tool’s performance have been positive, though some instructors say they’ve encountered bugs.
John Munro, an associate professor of business at the University of the Virgin Islands, thought the tool would be a good fit for his classes on computer applications like Microsoft Word for business students, as well as a class on business history that he teaches to a combination of in-person and remote students.
The traditional evaluation format had grown increasingly unhelpful to Munro -- he only got feedback after the semester was over, and the feedback he did get was from students worn out by finals and ready to move on. The one-to-10 grading scale on the existing form also didn’t lend itself to useful self-analysis, Munro said.
Once students have submitted their comments to Hubert at any time over a seven-day period, Munro lets Hubert do the tedious work he previously had to do himself.
“To me the benefit of the Hubert system is you don’t have administrative staff that have to transcribe and tabulate and go through the forms,” Munro said. “The person and time cost is reduced to essentially zero or very close.”
So far, Munro has learned that some students prefer not to have material “spoon-fed” to them -- he plans to cut back in future weeks. He’s looking forward to seeing whether the next round of feedback reflects recognition for actively addressing student concerns.
What It Could Do
More specialized uses for Hubert are also in the works. Samuel Adams, instructional technology specialist in the intensive English language program at Temple University, has tested the tool in training courses for teachers of English for speakers of other languages, and he hopes it will eventually work for students in those courses as well.
“I like that it just sort of condenses a lot of the lengthier sentences and explanations down into some key words,” Adams said. He also sometimes looks at individual responses to get a clearer sense of how the tool distills multiple sentences into a holistic report.
Adams said he’s struggled to use Hubert on his smartphone, and that the algorithm’s report on the comments hasn’t always aligned precisely with the actual comments. Still, he sees value in using the tool as developers continue to improve it.
With traditional evaluations, “it could be 500 to 700 points of data depending on how many courses and students we have per term. Having to process that over several weeks is pretty onerous,” Adams said. “If this could be something with a high degree of fidelity, I think it’s something worth exploring.”
