Learning to Adapt

Like most community colleges that enroll large numbers of low-income students, Essex County College has a serious graduation rate problem, with remedial math being a primary stumbling block.

Essex, located in Newark, N.J., had a graduation rate of 8 percent a couple years ago. About 85 percent of the college’s incoming students place into the lowest level of developmental math, and only 10 percent of those students end up completing a college-level math course.

So the college’s new leadership decided to give adaptive math software a whirl. Adaptive learning is an increasingly trendy form of instruction, typically featuring computerized courseware that adjusts to students’ learning styles and levels of achievement.

The college’s new president, Gale E. Gibson, made the call to spend $1.2 million on two new math labs for the project -- with work stations for 100 and 85 students. Essex got money from the Bill & Melinda Gates Foundation for some of the work, which included training for faculty members and the addition of new student support systems.

Essex chose ALEKS, an adaptive math learning system from McGraw-Hill, which experts say is one of the most battle-tested forms of adaptive software.

“ALEKS has zero multiple-choice questions,” said Douglas Walcerz, vice president of planning, research and assessment at Essex, who has led the adaptive work there. “It’s a very mature product. You don’t get software glitches.”

The college also went farther than some by designing an additional element of the curriculum for the two remedial and three entry-level math courses that feature ALEKS. It brought in John Hudesman, a psychologist and senior principal investigator at the Graduate Center of the City University of New York’s Center for Advanced Study in Education, to help design a “self-regulated” learning group approach for students.

This segment of the courses features two 50-minute sessions a week, where students discuss strategies that have worked for them. They also must maintain a workbook with learning goals, discussing them with instructors.

“I never believed that just giving them content was going to be enough,” said Walcerz. “We really wanted students to form a learning community.”

Disappointing Results

Yet despite all the hard work, and the money spent, the adaptive project at Essex isn’t working.

“At the end of the first year our pass rates were worse than the pass rates in our conventional courses,” Walcerz said. “That hurt.”

So far, roughly 1,000 students have taken the adaptive math courses. Typically about 50 percent of students pass the first level of developmental math at Essex. That number dropped to 35 percent in the adaptive version of the course in 2014. This year it crept closer to 50 percent, said Walcerz, but the adaptive version's success rate still lags behind that of the traditional version of the course.

Walcerz this week joined a group of experts on adaptive learning at a meeting the National Education Initiative hosted in Washington, D.C. He brought a somewhat glum tone to the discussion, which featured many executives from adaptive software vendors.

For example, during the meeting Walcerz called the project’s first semester “disastrous.” He also said the adaptive courses provide less “accountability.” That’s because students move through content at different paces and it’s harder to make sure they master concepts by a certain point. “There is no classwide mile post.”

Dror Ben-Naim attended the meeting. The founder and CEO of Smart Sparrow, an adaptive learning company, said Walcerz offered a “realistic” take on the hard work it takes to make adaptive learning work on a campus.

“There is this hope that this will be plug and play,” said Ben-Naim. “It’s really nonsense.”

He said adaptive learning “reconceptualizes the core product of higher education, which is the course.” As a result, it’s not a surprise that colleges like Essex would face some challenges while trying out the technology.

The news isn’t all bad for the adaptive courses at Essex. The college remains committed to the pilot project, which it is seeking to refine. Walcerz also stressed that ALEKS is holding up its end of the bargain.

“The technology does great,” he said. “I’m convinced it does a better job of delivering the content, given the constraints of having 24 students per instructor.”

Human elements pose the challenges, he said.

“Our problem is not content. Our problem is both student beliefs and behaviors,” said Walcerz.

For example, keeping students on task with the self-regulated workbooks has been a struggle for faculty members. About 90 percent of developmental math instructors at Essex are adjuncts. And the college leaned heavily on graduate students from nearby Rutgers University at Newark and the New Jersey Institute of Technology to teach parts of the adaptive courses during the first year.

That didn’t work, Walcerz said. Students often said they were on track in the sessions, even when that was far from the truth. So for the second year, Essex switched back to having its own faculty members run the self-regulated learning sessions.

“We underestimated the skill that you would need as a teacher to deliver that content,” he said.

Faculty buy-in has also been a challenge. In adaptive courses, instructors do not give lectures or teach in the traditional format. Instead, they circulate among students who are working on computer-based courseware, offering help when needed, much like tutors.

That feels like a job "below faculty status" for some instructors, Walcerz said. But he said others like the new format.

Staying the Course

Essex is hardly the first community college to use ALEKS or other adaptive programs in a computer lab-style course. This approach, called the “emporium” model, goes back decades and has had plenty of success. Many instructors at colleges where it has worked said they grew to like the model, and wouldn’t go back to the traditional format.

Phil Hill, an education technology consultant and expert on adaptive learning, traveled to Essex last year. He interviewed faculty members and students about the adaptive courses, including several who praised the format. (See below for a two-part web series Hill produced about his visit.)

The adjustment to adaptive can indeed be tough on faculty members, Hill said, because it forces them to rethink their role. But he has seen converts at several colleges. “For the ones who have done it, they like it,” he said.

Several students at Essex told Hill they appreciated that they had more ownership of the learning process in the adaptive courses, and more responsibility.

“They actually felt empowered to take control of their own educations,” he said. “They weren’t just learning math. It was a whole awakening of their education.”

Failure to improve developmental math success rates might not be an option for Essex. Hill called the challenge an “existential” one. That’s particularly true as colleges face increased pressure from policy makers at the state and federal levels to improve graduation rates.

For his part, Walcerz said Essex is sticking with the adaptive math project, which he said was largely his idea. And he said the college is even looking at other adaptive providers they can bring in to soup up the self-regulated pieces of the courses.

One plus for adaptive learning, he said, is that it requires a higher standard of mastery than a conventional course does.

“The students who get a C in the ALEKS-based math are prepared to succeed,” said Walcerz, which isn’t always the case with C students in conventional math courses. “Everybody has to learn the content, period.”

Another advantage is that adaptive courses provide data on student performance, which faculty members and administrators can use to tweak their approach with course design and teaching, what Ben-Naim called a “continuous improvement process.”

Over all, Walcerz said adaptive software is a tool that can help fix some problems. But improving student performance in developmental math is a particularly difficult challenge.

“You can’t learn for them,” he said. “It takes time and it’s hard.”