It's Not All Bad

Students are getting a better and more demanding education than scathing accounts like Academically Adrift suggest, but they and their instructors have plenty more work to do, a new study says.

“It’s lukewarm,” Corbin Campbell, the study’s author and an assistant professor of higher education at Columbia University's Teachers College, said of the academic rigor and teaching quality measured in the study.

While instructors were effective in teaching in-depth subject matter and the cognitive complexity of courses was about right, many students were neither expected to nor did participate in classes, most of which focused on “understanding and applying” rather than “analyzing and evaluating” course material.

The study included 150 class observations and syllabus reviews from one public and one private selective research institution, assessing academic rigor (quality of cognitive complexity, workload, standards and expectations) and teaching quality (in-depth subject matter ideas, using and transforming students’ prior knowledge, and supporting learning) in a wide range of undergraduate courses.

It’s the first pilot of Teachers College's College Educational Quality project, which aims to ultimately create alternative, less quantifiable measures of educational quality than standardized tests, surveys and other performance metrics. After a 10-institution study this fall and a national one two years after that, the benchmarks could be put into a database and referenced by people and institutions nationwide.

While there’s a lot of talk from opposite camps – the Academically Adrift supporters who say students learn next to nothing in college, and the scholars and politicians who say the U.S. has the finest higher education system in the world – Campbell’s research suggests the reality is somewhere in between, she said. (Academically Adrift is not a perfect comparison to the study, however, as it measures learning over students' entire time at college.)

“There are some strong educational processes happening at these institutions,” Campbell said, but she added that universities are “not maximizing their educational capacity.”

The researcher found that most (but still only 82 percent) of enrolled students attended class, which she called “fairly good news.” And in terms of cognitive complexity of assignments and coursework, professors are, on average, focusing on whether students can understand and apply information, and occasionally analyze it.

“When we think about college-level courses, that’s about where we want them to be,” Campbell said.

But the level of rigor does leave something to be desired. While class participation was typically noted as an expectation in the syllabus, it was not tied to grading and instructors expected only about half of students to be prepared and to participate (and only half did). The average class assigned several readings and while some were long and complex, most required only “a moderate amount” of work.

And while instructors scored “somewhat effective” and “effective” in teaching in-depth material by, for example, creating multiple representations of ideas and letting students engage thoughtfully, they were “ineffective” to “somewhat effective” in understanding and building on students’ prior knowledge, a key to learning. Instructors fared equally poorly at supporting learning and helping students who were challenged by course ideas.

Despite the fact that one of the universities charges much higher tuition, Campbell found no notable difference in teaching or learning between the two.

“With all of the conversation right now around cost, I think that’s interesting and worth noting,” she said. “Students who are going to the public institution on in-state tuition are probably paying less than they might pay at a private institution, and yet we’re seeing the similar education quality.”

The study did find differences between course types at each institution, although not by level, suggesting that introductory and advanced courses are being run the same way. But shorter and smaller classes scored higher on academic rigor and teaching quality than longer and bigger ones. And despite an apparent lack of student participation, courses with class activities and discussion scored statistically significantly higher on academic rigor and teaching quality, as did courses where students asked questions.

Campbell said that with a federal ratings system looming, it’ll be essential to have some kind of metric that actually measures the educational process, rather than just job and salary outcomes.

“I worry that that kind of an incentive structure would really force institutions to focus on being cheaper and faster,” she said, “and perhaps sacrificing educational quality.”