What Do We Know About This Spring's Remote Learning?

Welcome to "Transforming Teaching and Learning," a column that explores how colleges and professors are reimagining how they teach and how students learn. If you'd like to receive the free "Transforming Teaching and Learning" newsletter, please sign up here.

***

This spring COVID-19 forced hundreds of thousands of college instructors and millions of students to take their teaching and learning into a virtual realm most of them had not chosen and with which many of them were unfamiliar.

So how'd it go?

First, it’s important to say, it went. In other words, most faculty members made the switch adequately enough that most students were able to continue their educations rather than wash out. Given how consistently people love to say that higher education is stuck in its ways and can’t adapt, that alone might be considered a minor miracle. Professors adapted; colleges adapted. Most educations were not derailed.

Second, students and parents, as well as college leaders and professors, overwhelmingly believe that the learning experience was subpar. That would hardly be surprising, given the aforementioned lack of faculty and student experience and the fact that the pivot of in-person to remote teaching occurred with instructors having as little as a weekend and at most a week or 10 days to make the move.

It also recognizes that all parties involved have struggled through the last three months with varying degrees of personal and professional precariousness. Some have had trauma from coronavirus-related physical or mental health concerns or recession-driven economic woes.

But what do we really know about how it went?

Did less learning happen than it would have if students had remained in the physical classroom, as is widely asserted? Were students less engaged in their learning, and if so, was that because of more distractions in their lives or because the experience was less, well, engaging? Does the spring's experience give us meaningful insight into whether virtual forms of education can be effective?

And perhaps more importantly: What should we seek to learn about how it went, through surveys, data analysis or other means? And how should what we glean inform how colleges and universities educate their students this fall and beyond, given the likelihood that technology-enabled learning will remain central to the higher ed landscape in the COVID-19 era, and probably beyond?

***

Many critiques of the education colleges provided this spring (including one published elsewhere on this site today) referred to the instruction students received as online learning, rather than as emergency remote instruction, which is more accurate. The distinction may seem like hairsplitting to some, but I agree with others who say it's not.

While "online learning" can mean many different things, it has been practiced for more than two decades by many thousands of educators who have built up a large body of expertise and evidence that, done right, it can be effective.

By and large, what happened this spring wasn't that: it was legions of dedicated instructors doing their best to figure out how to deliver courses they had built for a physical classroom to a group of now-dispersed students, using whatever technology and often rudimentary pedagogical practices they (with help from their colleges' instructional designers and faculty development staff members) could master in a matter of days.

Faculty members and students alike were not well suited to thrive in that environment. A majority of faculty members had never taught an online course before this spring, and many had not had any training or preparation beyond what institutions were able to give them over spring break.

In normal times, students who've chosen to study online "know what they're signing up for," says Natasha Jankowski, executive director of the National Institute for Learning Outcomes Assessment. "If it's a synchronous class, you've committed to showing up at the same time each week and built time into your schedule so you can dedicate your attention and time to it."

That wasn't the case for most students thrust into remote learning this spring. "There was no guarantee they'd be available at the same time," Jankowski says. "Whether it was watching their kids, or picking up some extra work hours to pay bills, or caring for a loved one, education just may not have been the priority on their survival scale."

Even a course designed to be asynchronous may not have worked as intended for some students, says Jankowski, who is also a research associate professor in the department of education policy, organization and leadership at the University of Illinois at Urbana-Champaign. "Requiring people to create really cool videos of an oral presentation depends on people having good Wi-Fi access or technology," when some students' best Wi-Fi access was in a nearby parking lot. And many instructors put time limits on proctored exams, when there was "no guarantee I have three undisturbed hours in my house … We went into protection mode, security mode, instead of thinking about how we enable learning in a global pandemic."

For that reason, says Jillian Kinzie, associate director of Indiana University's Center for Postsecondary Research, "now is not a time to be judging anything about our effectiveness with online learning" based on this spring's crisis transition. It's also not a time to judge individual professors' efficacy in teaching, which is why many colleges have decided not to consider this spring's student evaluations of teaching in future decisions about tenure and promotion.

But just because it's unwise to judge the quality or potential of online learning by the rushed version of it most students encountered this spring doesn't mean we can't learn from the just-completed term, says Kinzie, whose Indiana center is home to the National Survey of Student Engagement, which gauges the perceptions of four-year-college students.

Early results from administrations of NSSE this spring show that next fall's incoming freshmen -- having had more experience with virtual learning than they otherwise would have had -- "realize how self-directed they need to be" to thrive in that setting, she says. "That's not a bad outcome from a slapdash approach to online instruction in K-12."

Those students "also have been tasked with inventorying their own capacity -- everything from software and hardware to their attention and personal capacities," Kinzie says. "How long can I sit and do this -- what's my attention span? Do I know how to access resources if I'm in a solely online experience? They've had to ask themselves all these questions."

Colleges should tap in to that as they consider how their own students fared this spring, Kinzie says. "Institutions could ask students to really inventory their skills, what they learned about themselves as learners, try to capture some of that," she says. "They could identify if a student had trouble paying attention because her house was noisy, because little brothers and sisters were bugging them so they couldn't get things done. Whether a student was able to find relevant resources when they couldn't walk to talk to a librarian or nudge a classmate in the next row. That is all valuable information for institutions and for individual learners."

To the widespread assertion that students "learned less" this spring, Kinzie asks a metaphorical "how would we know?"

"The professor's answer shouldn't be, because students scored lower on the final exam I produced for them that was the same exam I've been delivering for 30 years," she says. "There are just too many factors that could affect that -- taking on more hours at the local grocery store [to make up for a lost on-campus work-study job], caring for dependents."

"This semester has asterisks all the way down the list," Jankowski agrees. "Add the words 'in a global pandemic' to any question you might ask."

Kinzie hopes institutions will try to build off some of the creative new approaches to assessing student learning that emerged as professors had to experiment. "I know faculty who were really surprised, pleasantly surprised, by what students were able to produce in difficult circumstances because they still wanted to get something out if it," she says. "Let's look at what we did to allow students to demonstrate their learning in new ways, or more crafted by the connections they made with the content than the ones we were forcing them to demonstrate.

"Let's surface examples from faculty who had to resort to different forms of having students demonstrate a particular learning outcome, to show that students can be responsive when they're given a little freedom," she adds. "In the end, this could really help shift and reorient assessment practice to be much more about what the student is owning in the experience, rather than responding to the standard ways of expressing their learning."

***

Elsewhere on the Indiana University campus, Ben Motz, a research scientist in its department of psychological and brain sciences, is also on a quest to learn about this spring's learning.

As director of the university's e-learning research and practice lab, Motz is principal investigator on a new "Mega-Study of COVID-19 Impact in Higher Education." In conjunction with researchers at Ohio State University, Motz and his peers are surveying faculty members and students and analyzing learning analytics data from institutions in the Unizin Consortium of research universities with a goal described this way:

As our full nation's instructional faculty are suddenly forced to explore the contemporary online learning toolkit, and students are assigned to learn from whatever faculty cobble together, we have the obligation to understand the gaps that they discover, and how this impact is felt.

"There's a great deal of hunger for evidence of what the problems were so we can at least do due diligence of how we can fix them for the fall," Motz says. While many students and parents may have bemoaned the quality of the learning experience this spring, most also understand the crisis conditions under which it occurred. The expectations for virtual learning this fall will be higher, Motz says.

"It's as if faculty got a 'You pass Go and collect $200' card," he says, a Monopoly reference that might be lost on many of today's students. "The likelihood we'll get another favorable draw out of the community chest is low."

The study's goal is not to "evaluate online learning," Motz says, because "a large majority of faculty members and students didn't know what they were doing."

But the existence of a "field test" in which "100 percent of the target population" of students and faculty members "give it all a shot" created what Motz calls a "massive user study" both of the technology instructors and students used and the educational practices they employed.

On the technology side, it will be instructive to see how professors used various tools and how quickly they were able to adapt how they used them. "It almost doesn't matter whether a professor gets it right on the first try," Motz says of an instructor's interaction with a learning management system or videoconferencing platform or other technology. "Technology development is more of a sociological problem than a good use problem. If it takes you two years to become a power user [of a piece of technology], your product is broken."

Much more important is what researchers can glean about professors' interactions with students, and students' with course material and each other, Motz says.

While data from the faculty and student surveys are still being processed, very preliminary results reveal that students spent much of their time this spring reading textbooks and watching videos of instructors "giving the lecture he would otherwise have given," says Motz.

"It was spectacularly isolating," he adds. "The thing that was totally forgotten in this is any kind of contact among students or between students and faculty members. The faculty member seemed to feel the need to be a firehose of knowledge. The street was one way."

It isn't surprising that in the rush to transform courses in a hurry for a different mode of delivery that "the common response was to ignore those more interactive aspects of what online learning could be -- they just needed to survive," says Motz.

The good news about that is that's a fixable problem; for courses that remain virtual this fall (or that build virtual components into a hybrid model, as many institutions are considering), faculty members have more time to build in community-building elements that will make for a more engaging learning experience.

One other preliminary finding from the Unizin study suggests that instructors are game to try to improve their virtual teaching. Students who've responded to the survey say they are less likely to take online courses in the future, based on their experience this spring. But faculty members? "They're much more willing to teach online courses after the spring," Motz reports.

***

Jankowski of NILOA sees several key takeaways from the spring that she hopes will influence the faculty's approach to learning going forward.

First, early results of the association's own survey of assessment-related changes this spring shows that many instructors did not put student needs or issues of equity into account in their rush to transform their face-to-face courses for remote instruction. "A lot of people made quick decisions, then later asked, 'Did that work for you?'"

Very quickly, though, the diversity in students' needs and situations became "starkly raised" for instructors, Jankowski says. In normal times on a campus, students turn to various student affairs offices to deal with problems or difficult situations.

But in the pivot to remote learning, "the main touch point that students had with the institution was with faculty, and they were getting bombarded with questions about mental health, medical things." On campus, a professor might have referred a student to student affairs, says Jankowski, but with urgent requests, many instructors "ended up having to get support from student affairs professionals for what they do holistically for students."

"I'd like to think professors came away from this spring with a better understanding of the whole student -- not just for the time they show up in my class, but the fact that they have layers of things that are going on. They're not just a student, but a caregiver, or a foster youth … I would like to see us not be surprised about our students."

Jankowski also, unsurprisingly, sees opportunity for instructors to emerge from their experience this spring with a heightened sense of the importance of how they assess students' learning.

She says her anecdotal sense is that faculty members who had built their in-person courses very specifically around a set of learning outcomes "understood the value of that when they made the pivot" to remote learning, when many of them reconsidered their expectations for the amount of work students could do given everything else they were juggling.

"It gave them an ability to home in on the most important learning for the end of that term," Jankowski says. As professors considered what assignments to keep and which to ditch, those with a clear sense of the course's goals had an easier time deciding "what do I need to have my students focus on, what are the most important parts of what students need to learn."

"One question for future is whether the importance of learning outcomes and assessment as a design tool carries over and permeates how we build courses," she adds.

Jankowski says she saw professors adapt in another way that heartens her -- by shaping their assignments in response to what students were encountering day to day.

A math professor who asked students to graph their internet speed over time, to gauge how it might affect their learning. A history instructor who incorporated the 1918 flu into the course plan. Psychology faculty members who asked students to watch a movie with the family members they were holed up with to understand the differing prisms through which they viewed it.

To the widespread assumption that students learned less during this time, Jankowski acknowledges that that's a possibility. But maybe it was just a different kind of learning, she says -- "maybe it became more poignant because it was relevant to how I was living."

She cites another example of music students who used video platforms like YouTube or Flipgrid for group recitals. "That wasn't the faculty that figured it out -- it was the students. It could benefit us to keep in mind that they can be co-creators, and they might have really good ideas if we make clear the outcomes we're trying to get to."