Using Data to Drive Performance

ST. PAUL, MINN. -- Pockets of experimentation and potential change are cropping up all over the place in higher education. Here, it's the OpenCourseWare movement. There, colleges are adopting "just in time" remediation. And over here, some states are changing funding formulas to reward institutions for graduating students rather than merely enrolling them.

But what do the individual innovations amount to? Do they point the way to the sort of transformative change that, given the likelihood of constrained budgets going forward, is probably necessary if higher education is to not only sustain the current level of postsecondary attainment in the country, but increase it in the way many policy experts believe is needed?

Even longtime advocates of higher education appear to be coming around to the conclusion that the status quo won't suffice. But acknowledging that fact is a far cry from identifying a framework that might lead to such a transformed future. It was in recognition of the latter goal that an unusually diverse group of college administrators and professors, higher education analysts, state officials, and others gathered here last week to talk about how to use data to provoke change and improve performance in higher education.

The event, the Action Analytics Symposium, was unusual in several ways. (One way in which it was not exceptional was that it was sponsored by the Bill & Melinda Gates Foundation; Gates and the Lumina Foundation for Education are the muscle, financial and otherwise, behind many if not most gatherings of would-be change agents in higher education these days.)

Chief among the meeting's distinctions was that it was co-sponsored by two institutions of very different types: the Minnesota State Colleges and Universities, a 29-institution system of public two-year and four-year colleges, and Capella University, a for-profit online institution that happens to be based in Minneapolis. The for-profit and nonprofit sectors tend to clash more than collaborate, a reality that Michael J. Offerman, Capella's interim president, and James H. McCormick, MnSCU's chancellor, are trying to change.

Offerman, Linda Baer, MnSCU's senior vice chancellor for academic and student affairs, and Donald Norris, a consultant, framed the concept behind Action Analytics, which they define as "a multi-dimensional effort aimed squarely at fundamental and strategic education reform in the context of massive economic, social, and technological change... [A]ction analytics strategically uses data, statistics, predictive modeling and visualization to promote student success, achieve institutional efficiency, and demonstrate transparency."

That definition may have made your eyes glaze, and if so, you weren't alone; like many a high-concept buzzword, "action analytics" seemed to mean different things to many of the 100 participants in the meeting. To the extent that there was a consensus, though, it might be summed up in the idea that to avoid having undesirable change thrust upon them by government regulation or a collapse of their economic models, college leaders must not only collect and make public much better data about how they perform, but actually use it to improve that performance.

Talk of data and performance raises the (terrifying) specter, for many a faculty member, of No Child Left Behind-style accountability in higher education; for privacy advocates, it may provoke fears of a "cradle to grave" system of data about individual students. And there was indeed much discussion of how student data systems, linking a state's colleges back to its public schools and forward to the work force to gauge employment outcomes, can "transform higher education into an information-informed sector," as Aimee Guidera of the Data Quality Campaign put it.

With regard to the money that the Obama administration (with its stimulus and Race to the Top funds) and the foundation world are pouring into state data systems, "there is an incredible window of opportunity that will never, ever be repeated," Guidera said. "There's a totally new game in town. How do we take advantage of it?"

While she and many of the other speakers and participants at last week's meeting are strong believers in the role that technology and data can play in bringing about change in higher education, they sought to make clear that while external calls for accountability may prove helpful in prodding recalcitrant colleges to change, their ultimate goal is to help institutions help their students, by "bringing data down into the learning moment," in the words of Mark David Milliron.

"We're not talking about connecting the world and having one big database in the sky," Guidera said. "Think of this as a flashlight instead of a hammer," to help identify practices that make instructors more effective at keeping students on track.

"Can we, in the moment [students] need support, get them the support we know they need?" asked Milliron, deputy director for postsecondary improvement at the Gates foundation, who urged what he called "targeted, triggered and tailored" information about students. Is a college that knows that a student is late in making a tuition payment "immediately getting that student the information or support she needs?" What if it identifies a student who has just dropped one course and asked an academic adviser about dropping a second. "Let's figure out how we can intervene," Milliron said.

Or "let's say you found out that 89 percent of a high school's students got A's and B's in calculus, but 25-50 percent that scored 'proficient' on an exit test in math ended up needing remediation" at a public college in the state," Guidera said. "Something clearly is not aligned, but you'd have to act on that to figure out what's going on there. Are the [exit and placement] tests not aligned? Is the [high school's] curriculum not the right one?"

While many people at the meeting seemed to find the prospect of that sort of on-the-ground, student-focused information immensely appealing, it seemed far less obvious how to generate the kind of coordinated, large-scale transformation of higher education that might be necessary to achieve the Obama administration's goal of radically increasing postsecondary attainment in a decade. Efforts by individual institutions, or even innovative steps by 50 states, are likely to prove inadequate in a higher education world in which students move freely among institutions, states, and postsecondary sectors, several speakers suggested.

"There are a lot of emerging technologies that we tend to employ piecemeal, and there's a lack of uniformity and ability to bring those together," said Charles Lenth, vice president for policy analysis and academic policy at the State Higher Education Executive Officers group. "There's no entity that can take hold of this and move it forward, to create a framework across states to help develop more comprehensive approaches."

Those behind Action Analytics wouldn't dare assert that they have a clear solution to that problem, but they left last week's meeting vowing to keep attacking it. They plan to convene national experts in a Web-based community of practice, and to test out concepts locally in the Twin Cities, involving not just the host institutions but the mammoth University of Minnesota, too.

While some participants expressed concern that the continuing economic woes in the states would discourage progress on this front -- since "bad times sometimes erode away what was innovative," said Baer of MnSCU -- others argued that the combination of economic necessity and continued external pressure from federal and state policy makers would compel college leaders to find new ways to improve their own performance.

"If we don't," said Guidera of the Data Quality Campaign, "we will go the way of newsprint."