Higher Education Has a Data Problem

The truest measure of any technology’s value is the return the consumer gets on their investment. On behalf of their students, colleges and universities are today spending billions on new digital tools designed to propel the rapid shift to remote and online learning. But up to this point, the returns have fallen short.

According to a new survey, many students have begun to doubt the underlying value of a degree, with fewer than one in five responding that the learning experience is worth the cost. Today, professors, administrators and their private-sector technology partners need to find better ways to harness the range of new tools at our disposal to improve students’ learning and educational experiences in real time.

Never before have colleges and universities needed to demonstrate so clearly the value of what they provide students. That’s in no small part because higher education institutions now compete more directly with each other on educational efficacy. During the “sage on the stage” era, marketing materials and glossy brochures often highlighted campus life. That’s no longer the only salient criterion. Students are now more prone to judge professors and institutions on the basis of whether they will come away having attained knowledge and skills relevant for their long-term career success.

The good news is that, to a degree unavailable just a few years ago, technology and the data it generates hold the potential to empower instructors to give students feedback not only when a course is completed but also while they’re engaged in their studies, enabling them to glean more from the learning experience. But to maximize the return from those digital tools, those shaping and using technology need to think proactively, intentionally and creatively about how data are generated and how they are best integrated into instruction.

Using the Wisdom Gleaned From Data

Professors today are inundated with technology equipped with the potential to help them hone their craft. For example, if a technology can cull through quiz results to identify which concepts remain hazy in students’ minds, instructors can quickly discern which of their various lesson plans is most effective.

But perhaps more important, technological tools, if properly conceived and used, can now deliver small (personalized) data to help instructors flag students who are struggling to understand the coursework, stay engaged or maintain their motivation. For instance, online homework assignments should be able to reveal who is lost in the material without requiring instructors to invest hours analyzing individual submissions. The data generated from in-class activities like quizzes, polls and discussions should provide insights into student comprehension and levels of participation. Attendance records can pinpoint which students are buckling under their course load. And if all these varied and various data points are considered together, they should provide actionable information.

In other words, if created and collected conscientiously, data can help educators intervene in short order by helping them discern when to assign a tutor, recommend a remedial reading or simply invite a student to revisit a core concept during office hours.

But therein lies a challenge: when educators use multiple tech tools, they contend with the challenge of managing and understanding siloed, disaggregated data. As a result, instructors receive only fragments of the overall picture, leaving many to feel overwhelmed by multiple dashboards and reports. Unable to piece together all the different indicators, instructors struggle to glean real wisdom, let alone adjust to a student’s needs. In an environment where they compete directly with each other on how much students learn through their coursework, colleges and universities cannot afford that lost opportunity.

Taking a Holistic View of Student Performance

Some institutions have begun tackling this challenge with their outside technological partners. Braiding together disparate sources of student-centered data, they are collaboratively finding ways to produce holistic views of each student’s engagement and performance. What’s more, they’re almost immediately finding that a comprehensive data picture can quickly provide a profound value-add for administrators focused on bottom-line educational results.

Key to their progress, however, is the cooperative nature of the partnership. Neither university administrators nor their technological partners can do it alone. If they are going to serve students and instructors to the fullest degree, vendor partners need to produce reliable data, and institutions need to aggregate them.

Over the past few months, Top Hat and Unizin, a technology consortium of higher ed institutions, have collaboratively developed a rich data integration of Top Hat’s digital courseware platform into the Unizin Data Platform, or UDP. Using data standards, the integration combines data generated in Top Hat with the unified, commonly modeled teaching and learning data that the UDP aggregates from other learning tools, including learning management systems. Suddenly, instructors have had at their fingertips comprehensive data capable of pointing them to better educational interventions for each student. As Bart Pursel, Pennsylvania State University’s assistant director of teaching and learning with technology innovation, recently explained to us, the data integration project “gives professors a more complete picture of the student journey so that they can conduct timely and targeted outreach.” And that’s exactly the point.

When Penn State instructors and administrators first moved to remote teaching at the start of the pandemic, the university relied heavily on its learning management system. But in short order, administrators realized they wanted instructors to engage their students not only through the LMS, but before, during and after class, as well. So, they turned to Top Hat to help them build interactions and engagement directly into existing elements of each course -- lectures, digital textbooks, class discussions and more. The challenge, absent the partnerships between the university and Unizin, would have been weaving data from the various technological interventions together. By using the UDP, however, learning in Top Hat and the LMS could be captured with comprehensive data capable of painting complete portraits of individual student journeys, empowering instructors to adjust to each student’s successes and struggles.

The Top Hat-Unizin collaboration has enabled us to validate a number of hypotheses about how institutions, educators and technology companies can work most effectively to create, collect and use learning data that enables instructors to support better learner outcomes. They include:

• Institutions must design their data strategies with actionable insights in mind. The purpose of an educational data strategy is to inform effective course design and instruction in the immediate term. There’s no sense collecting data that don’t bear on the meaningful decisions that instructors make when working to support individual student success and deliver personalized learning experiences at scale. Developing a strategy also clarifies what kind of learning data you require and why.
• Vendors must focus on producing high-quality data designed to solve a problem. Partner vendors must think about generating data from the perspective of the needs of the instructor, instructional designer and other stakeholders. That is, they should view the data they generate with an eye toward improved teaching and learning experiences. Valuable data must be: 1) comprehensive, describing everything relevant to addressing a need or problem, 2) correct, accurately reflecting student performance and 3) complete. Vendors should also analyze and process data in a responsible way and ensure security and privacy are part of the design process.
• Institutions and vendors must work together to guarantee students derive value from their education. The temptation when trying to derive wisdom from data is to ask at the outset what precise data are already available and then make do. That’s a mistake. Rather, institutions like Penn State and its partners, such as Unizin and Top Hat, should first define the problems they need to solve -- and only then should they begin to focus on how to create and leverage data toward that end.

Despite emerging concerns that higher education is not worth the cost, studies prove that college degrees will remain a key differentiator in the marketplace for tomorrow’s high-paying jobs. Technology is poised to help institutions of higher education deliver, but to take advantage of that opportunity, universities and their partners need to do more than collect data for long-term evaluation. By unlocking the power of digitized information to elevate learning experiences, administrators and educators can deliver meaningful value to learners. Together, they can keep students motivated and highlight how their learning journey will help them succeed beyond their college experience.