Predicting Success

Many colleges are flirting with how to use predictive analytics to boost their graduation rates. But big data is often just a flashy way to spend money on reports aimed at administrators, argue the leaders of Civitas Learning.

They say data science works best when converted to practical tools that faculty members, advisers and students themselves can use. And the company, which is a relatively new player in education technology, so far has signed up more than 25 institutions -- including statewide systems and national chains of campuses -- to give its products a whirl.

Mark David Milliron is co-founder and chief learning officer of the Austin-based Civitas. A former high-profile official at the Bill and Melinda Gates Foundation, he moved to the company last summer from Western Governors University’s Texas branch. Milliron’s role with Civitas says a lot about the upstart company’s focus on the national college completion push.

“We’re trying to help institutions that want to improve student success,” he said.

The various cloud-based apps from Civitas are user-friendly and can be customized by institutions, Milliron said, which is part of the company’s “low friction” approach. That means colleges can use their analytics tools differently, focusing on specific audiences. And they can do so without spending a lot of money on training.

Even so, Civitas might make some in higher education nervous. There are other established companies in the hot "adaptive learning" landscape, such as Knewton or CCKF. But the pitch from Civitas is that it provides a broad "organizing layer" of data from many sources and gets it to the front lines at an institution. The information it gleans can be used to give a better picture of how students are faring in real time.

For example, Civitas could measure various aspects of student performance in individual courses, even comparing how instructors fare across sections.

'Rate of Change'

Strayer University, a major for-profit chain, is doing exactly that.

Karl McDonnell, Strayer’s chief executive officer, said the company has used Civitas to measure the “impact an individual faculty member has on the engagement of students.”

Civitas is able to control for differences among student preparation and other variables. As a result McDonnell said Strayer creates a “real-time dashboard” to inform instructors -- and compare them with all other faculty members -- as courses progress by evaluating the “rate of change” in student engagement, which encompasses everything from how often they contribute in class to their performance on homework assignments.

Students get an entering engagement score in each course. If instructors lag behind their peers in making progress with those indicators, the university reaches out to help get those faculty members back on track, McDonnell said.

For example, Strayer studies how top-performing instructors get results. They use that information to reach out to faculty members with personalized messages, and also send a weekly email to all university instructors with three or four tips on how better reach students.

Milliron approves of how Strayer has employed Civitas. But he said any system that uses detailed analytics has the potential to be misused. To prevent that, Civitas works hard to help institutions get the most out of its tools in a non-prescriptive way, said Milliron, with a central goal of empowering “student success scientists” on campuses.

The company provides real-time information about student progress to faculty members, Milliron said, which they can then choose to use to help "guide their daily instruction." (Note: This paragraph has been added to a previous version of the article to clarify Milliron's position on the analytics tool's potential uses.)

Harnessing big data to rate instructors almost certainly would generate controversy, among faculty groups and others. It already has in K-12, mostly notably with the recent furor over New York City's teacher ratings.

Rudy Fichtenbaum, a professor of economics at Wright State University and president of the American Association of University Professors (AAUP), said he is "deeply skeptical" about whether Civitas or other learning analytics tools could do a good job of rating instructors. One key reason, he said, is that many factors influencing student engagement are out of faculty members' control, such as distractions students face outside the classroom.

Instructors at Strayer have made good use of the Civitas dashboards, McDonnell said, taking ownership of the tools without fearing a big brother is “looking over their shoulder.” But that might’ve been trickier for Strayer to achieve if it had tenure or a faculty union, he acknowledged.

“The faculty are absolutely central to what happens with a student,” said McDonnell. “They like being able to see the impact they have.”

Working with Community Colleges

Civitas got started about 18 months ago with six “beta” partner institutions. It has since begun ramping up, while applying lessons learned from the initial group.

The company’s current client list runs the gamut. It includes the Kentucky Community and Technical College System and a wide range of community colleges, research universities like the University of Arizona and University of Texas at Austin, for-profit chains like Strayer University and DeVry University, and a historically black institution – Morehouse College.

Several other companies offer completion-oriented tools. For example, InsideTrack has long been active in the space, albeit with a different product.  The Bay Area-based company offers outsourced student coaching and advising services, mostly to four-year institutions.

Both InsideTrack and Civitas have had success in the community college sector. That might surprise some, because although many two-year institutions could use help with student retention, most lack the money to bring in outside firms.

InsideTrack now works with community colleges in Alabama, Montana and the statewide system in Tennessee, among others, said David Jarrat, the company’s vice president of marketing.

Community colleges face “greater and greater incentive for them to improve their retention,” he said. And performance-based funding formulas in Tennessee and many other states help make it “financially viable” to hire InsideTrack or other companies.

Austin Community College is one of the first six institutions to contract with Civitas, thanks in part to a $100,000 grant from the Gates Foundation.

The large two-year college had been feeling plenty of pressure to improve its student retention and completion rates. A Texas business group in 2011 even paid for a billboard to try to shame the college for its then-8 percent graduate rate (a number based solely on first-time, full-time students, which account for a small amount of total enrollment at the college).

As is the case at many community colleges, Austin’s student advisers are often too overwhelmed to help large numbers of first-generation and low-income students. The college has just 300 advisers on staff to serve its 43,000 students.

That’s where Civitas comes in, said Melissa Biegert, a project director at the college.

The company’s "Degree Map" application is an “immediately visual” and “very handy” way of tracking student progress, said Biegert, who leads the college’s Integrated Planning & Advising Services (IPAS) project.

“The counselors spend an inordinate amount of time gathering paperwork,” she said. And Civitas helps them reduce the clutter by combining “a lot of resources that are scattered all over the place.”

For example, Civitas puts the entire course catalog into its dashboards, which show how much progress students have made toward their credentials. The tools also help students change majors, Biegert said, without losing credits.

In addition to customizing data for all of its advisers, the college recently experimented with offering Civitas apps directly to 1,500 students. By the end of the semester the college plans to expand that use to all its students.

Biegert said the next step is to work with faculty on predictive analytics. The goal is to help instructors do more advising. (The college is not planning to compare faculty performance, however.)

“This gives them a tool that makes that easier,” she said.

However, using predictive analytics in student advising can create challenges, according to Civitas clients and Milliron. Giving students more information about their academic progress is obviously a good thing, on balance. But colleges need to be mindful about discouraging students who can succeed.

For example, Milliron said institutions probably don’t want to use Civitas to send a “flashing red light” to students telling them they have slim odds of graduating. Instead the company works with colleges to create short-term goals for students who need extra help.

Strayer has used those “suggested interventions” with at-risk students as part of its collaboration with Civitas. But McDonnell said sometimes the university looks at Civitas data and tells students they lack the preparation or engagement to succeed. The university then urges those students to leave before taking on unnecessary debt.

“It’s a lose-lose if a student continues to try,” he said, “but they just don’t have the academic background.”