Dealing With Students’ Learning Loss: The Key Podcast

Learning loss -- the idea that pandemic-era students failed to stay on the learning trajectory they would have been expected to follow in normal times -- is much discussed in elementary and secondary education. But it isn’t talked about much in higher education, which makes sense, given that colleges and universities collectively spend a lot less time than elementary and high schools trying to quantify students’ learning.

But in survey after survey the past 18 months, including those done by Inside Higher Ed and College Pulse as part of the Student Voice project, students and faculty members alike consistently say they believe students have learned less than they usually do.

A recent episode of Inside Higher Ed’s news and analysis podcast, The Key, explored what colleges and universities face as they welcome students back into their physical classrooms this fall, and how professors and staff members who work with students might go about understanding which students have been set back and in what ways, and how to get them back on track.

The conversation featured two guests: Natasha Jankowski, former executive director of the National Institute for Learning Outcomes Assessment, now a consultant on student learning and a lecturer at New England College; and Ereka R. Williams, associate provost for academic strategy and institutional effectiveness at Winston-Salem State University.

An edited version of the podcast appears below, beginning with the discussion with Jankowski.

Inside Higher Ed: How would you describe learning loss, and what do we know about whether and how much it has occurred, both for incoming college students and for those who were already enrolled in college before the pandemic?

Jankowski: I’m going to answer that by sort of providing a little bit of context. We had this past year and a half where our students and our faculty did not have the luxury or the privilege to just be students and learners, or teachers. We also know that our faculty, while incredibly engaged in trying, were maybe not at their best because of pivots to online and shifting between “How am I modifying for some hybrid in-person while also still doing remote?”

And while it got better, we know that the pressures and the pandemic and all of that -- homeschooling, day care, concerns about health -- that never went away. So you have this constant stressor going for this period of time in which we’re hoping learning unfolds.

We have information from students in the College Pulse survey, and over half said that they learned less. And they felt unprepared for college. Sophomores actually reported having the hardest transition, because if you think about it, if you were a sophomore now at the time this survey was going on, you were a freshman for about a full semester. You got to experience that institution before the pivot happened, the pivot semester. And so how you engage in your social groups, your relationship with the institution and teachers, all of that has this sudden upheaval. So we know the students say they learned less. Faculty, when they’re asked, agree.

We also know that students in ongoing surveys reported that they felt unmotivated, that they were distracted, that they had a hard time finding a place to study and learn, that they’re concerned about mental health, that they feel behind.

So we know that learning loss happened. That’s learning in the sense of, if we had certain things we want you to take away, the likelihood of you as a student getting that, and me as a faculty member teaching it to you really well, was unlikely. More often than not, you probably got something, but not all of it. If I as an institution have designed some programmatic path of learning that grows for my students, then I need to figure out where you are on that path, and how to reinforce what I need you to engage with and where you are as a learner.

Inside Higher Ed: In normal times, what were the tools that institutions used to gauge where students were on that path, be it incoming students or continuing students? My sense is that tools were pretty darn imperfect to begin with and may not be up to the task in these even more difficult and challenging times.

Jankowski: While we had some tools, they’re imperfect tools. We use a variety of things in our institutions. If you’re incoming, we have placement testing. Admissions, we’re looking at your SAT, ACT scores, those kinds of things. We know now institutions are saying, “We’ll waive that,” or “we’re not engaging with it.” The role of what a placement test can do and where we put you is something different, especially for our incoming students that have had a very different end-of-high-school experience.

We also had assumptions about course progressions and course completion, so that if you got through a course, we’re assuming you learned something from that course, and what that course was supposed to do, whether it’s gen ed or major-specific. Now we’re in a place where that’s under question.

Inside Higher Ed: What was behind your assertion that college courses may not have done what they were designed to do? Was it because they were being taught in different if not lesser ways because of the pandemic, or the fact that both students and instructors were maybe at less than full capacity?

Jankowski: There’s an idea that we have some intentional design behind the learning experiences we ask our students to do, and that if you’ve gone through a course or some kind of educational experience, you’ve got a specific learning outcome from it. Now, as someone who has studied assessment over time, that’s a questionable assumption at best. I think we have good intentions, but if it’s actually our design, I’m not so sure.

The National Institute did a survey at the start after the spring semester [2020] where everyone went remote. It ended being a great “return to assessment basics” moment, where faculty members were going, “Well, what is actually the purpose of my course? And what do I really need you to learn from it and what’s the assignment and the task that I’m getting at?” So that’s a really positive pedagogical conversation, which I appreciate. But the larger question of “have we ever been in a place as an institution of education to say that when you graduate with a credential or a degree, you actually know and can do these things, and we have evidence and proof of it as we’re going along” -- no, not as much.

Inside Higher Ed: It’s really hard in any of these conversations to talk about students collectively. So break down a little bit for us the sort of different ways that you’re thinking about different groups of students, based on where they were during the pandemic. What are the potentially different ways we should be thinking about those different groups?

Jankowski: If we start to parse our student population, most of our students, while they engage with technology, don’t know how to use it to learn. And so while our faculty were figuring out how to do this online, our students were also figuring out how to do this online, and then you also had some subsets of students that had very large technology barriers -- access to technology, access to internet, access to Wi-Fi -- and most of our surveys were done via email, so if you can’t get online, I don’t find you and I can’t figure out what you need.

We also then have our students of color that were watching unfold a racial reckoning, and how am I supposed to learn when I’m in constant sort of fear and concern? We know from neuroscience and cognition literature that learning under stress you do not retain, if you can even learn at that instant. And so you have that undercurrent going throughout a pandemic.

Inside Higher Ed: And they were disproportionately affected by the pandemic itself in most places in the country.

Jankowski: Yes, and concern for family members, and financial security. And so within any of those subgroups, the question is not even what kind of learning loss are we looking at, and more, can we even assume that learning happened? Or was it a matter of survival and just checking boxes?

Inside Higher Ed: And we saw how many students just didn’t continue. Those were at obviously at an extreme, but it wasn’t a small extreme. We lost half a million students, probably close to it, who for one reason or another either didn’t start or didn’t continue.

Jankowski: And for institutions, if we put in too high of a hurdle around learning recovery, we don’t get those students back. That’s a group of students we’ve lost. Institutions of higher education really need to look at their student populations and who they serve, and how their students experienced this past year and a half very differently.

We also have some groups of incoming freshmen that didn’t have closure around high school. The K-12 sort of remote learning experience, it’s very different from the pivot that that went on in higher ed. We at least had more tools, and maybe engagement with online learning, than some of our K-12 partners had, so they’re going to definitely need not just orientation to institutions but to expectations of engaging in learning and behaviors, and knowledge and skills at an institution level.

We also had a freshman class that started remote. They started last fall, or they had a very hybrid experience behind Plexiglas and masks. And they’re now becoming sophomores and need an orientation to the institution -- to [things like] understanding of major choices. While we don’t normally think about your sophomores in that way, that’s a crucial retention learning group as well.

Then we also had our freshman class during the pivot that are going to be incoming juniors. They got sent home and have had a very mixed relationship with their institution, so we need to think about how are we reorienting and reconnecting with them.

And if you were doing a two-year or a shorter-term credential, you’re gone already. What experience have you had with us and how do we need to think about that alumni connection so that we still have a lifelong learner partner and not someone that’s just had this very surreal experience?

Inside Higher Ed: That’s a seriously scrambled picture. So if you’re in academic affairs or student affairs, or if you’re a professor, what are the strategies or tools or approaches you should be thinking about to try to make sense of this puzzle and to try to make sure it’s personalized or targeted enough that students at their different levels are getting what they need to remain or to get back on track?

Jankowski: I’ll start with some ideas for institutions. We’ve paid a lot of meaningful attention on our faculty and what they need, and less so on our staff, who are the ones that are really going to be picking up and carrying that sort of personal retouch [with students]. So for an institution, really start with empathy for your faculty and your staff, and with care. If we start mandating more things at this point in time, that’s going to be less than ideal.

There is an opportunity to co-construct there. We know we need to offer multiple orientations to our students -- what might that look like? While that might normally be a student affairs issue, how is this something we can do together? We also need to think about how we are orienting online learning for students. Students who have been through it, they could have locked in bad behaviors in terms of how they engage with online learning, that happened out of a quick need, but now they’ve been doing it long enough that it’s an ingrained behavioral pattern.

Inside Higher Ed: If institutions or instructors assume that there’s been some kind of learning loss, do you envision them responding by paring back what they try to accomplish in terms of volume of content or instituting policies that provide more flexible grading or take other approaches like that? Or do you anticipate a push to turn to the rigor of before?

Jankowski: That’s a definite caution to avoid, because the ripples from this will persist. This isn’t done and we can’t just sort of be like, “Well, we’re back to campus and normal.” If we start putting in those now that you’re back at this institution full-time, no flexibility at all on this assignment, we’re going to cause more harm than good in that space. This is a great opportunity for institutions to really consider students as partners in our process of identifying issues and developing solutions.

And it’s honest conversations -- where is everyone at and what can and can’t you do right now? Give me your timeline of when you can submit work, based on what your current schedule is and let’s figure it out. That’s a great self-reflective tool to engage in. Do I know how much time it takes me to do something as a learner? It’s a wonderful cognitive exercise that people should do.

But I worry in hearing some conversations at institutions that they’re rethinking the role of prerequisites to be back to more of a gatekeeper, and saying, “Well, if I put a bunch of course prereqs in the way, that will give me quality control.” We need to not be making policies about blocking students out. We need to really be making policies that lead with lifting learning up, and thinking differently about the role of things like a prerequisite, thinking differently about how I figure out where students are the first day they come into a course, and engage, and who I share that information with to help future planning. It becomes much of a team sport, because it’s really going to take the entirety of our institution to help our students move through this. And also not to have our faculty and our staff feel like this is impossible for me to solve on my own.

Inside Higher Ed: If you are an individual instructor, depending on how much help and guidance you’re getting from your institution, what are some approaches, what are some ways you would encourage them to be thinking about all those things, judging where their students are starting and how to move them along, especially if they’re in a “lesser place” than you might be accustomed to them being?

Jankowski: I would start with, take your summative assessment hat and go hang it up for a while and put your formative one on, and really engage in that formative development assessment. Go back and dust off your Tom Angelo and Pat Cross book on asking quick, easy things to do to get handles on where students are as they’re consistently moving through that learning. If we lock this into a couple of summative points -- midterm, final -- we are not going to be able to move the needle in the ways that our students will need us to address and get over this gap.

If you’re not a fan of classroom assessment techniques, Small Teaching, by James Lang, is the same sort of idea. Go find your center for teaching and learning person, and think about the pedagogical approaches that help you in that formative assessment place. We don’t want to take a gotcha stance on learning; you want to take a very developmental stance on learning.

Also think about partnering with our students. Peter Felten just did some great work on relationship-rich education. And one of the things that we had heard in the National Institute for Learning Outcomes Assessment survey was faculty and students both reporting that realizing they’re whole people … led to a better relationship where they were able to share more, learn better together and create an environment where I can speed up in that process because I have trust we’re moving and I can get it done. And that, again, moves us away from more of a gotcha, gatekeeping function and more into a “we’re an environment of learning.”

Inside Higher Ed: Are there things that make you more optimistic than you might have been, and were there silver linings from the pandemic and the way it changed learning that might help institutions and instructors and students address whatever learning loss there might have been?

Jankowski: For sure. The realization of who students actually are on the part of faculty was great. The understanding of the need for transparency -- I really as a faculty member have to tell you as a student why I’m asking you to do this, what I want you to get from it, and thinking differently about how to apply it, and asking my students to say, “What’s going on in your life right now that you can use to engage with this?”

There were, for instance, math faculty sharing assignments on having students run tests of their internet speeds at their house, and then graph it, focused on quantitative reasoning skills and all of that. But that’s immediately applicable to something that I’m engaged in doing, and making learning relevant was very crucial and helpful.

But also realizing that our students are people. They’re not just students, and they’re not just there to learn. They’re learning in spite of all of these other things, or as all the rest of these things are going on.

I remain optimistic that faculty in the pivot, and persisting over time, have a new appreciation for student affairs professionals that they did not have before. If I’m the person you see, I’m now getting all these questions: “I’m having mental health issues. I’m having this home situation.” How do I engage with that?

On one hand, that helps me as a professor understand you as a person, but it also has me thinking, oh my gosh, student affairs friends, I get how this actually is a partnership in support of learning. Our students can’t learn if they’re hungry. Our students can’t learn if they’re not in a mental space to do so. And so if some of our solutions to really addressing learning loss are going to be holistic, that needs academic and student affairs working in partnership. I hope that newfound appreciation persists -- and we don’t sort of go back into silos and our discipline homes, and forget that we each bring strengths to really enabling and supporting learning.

Inside Higher Ed: You talked about the increased understanding [between students and professors], and maybe that leading to better relationships and more respect. The one thing that might have mitigated against that was perception on the faculty side about there being increased cheating by students. I’m curious whether you think the sense the faculty members have that there was greater academic misconduct and dishonesty during the pandemic … will have lasting impacts, or will people sort of write that off as having been of the moment?

Jankowski: There’s a couple of ways that it could go. One is you get faculty watching students take a test. And I feel better about it and now maybe my concern about cheating goes down. But the opposite of that is to say I’m now expecting you all to cheat, and even though you’re doing it in front of me, I’m assuming you figured out a workaround, and so now I start spending a lot of time and energy fielding up ways in which I can block that, and I’m blocking cellphones and I’m turning off internet in my room

If we do that, we lose sight of several things. One, students even reported that they cheated more, and we saw cheating go up. That’s legit. But it was a pandemic -- that’s not oversurprising that you would see things related to each other. But if we didn’t respond to these situations with empathy and humanity, and we instead answered with “Still take this high-stakes exam …”

Inside Higher Ed: And do it in front of a camera …

Jankowski: And do it in front of a camera and don’t blink too much, don’t have a dog walk behind you, as though we can really find those places to do it, are we surprised the students responded in that way? And I think it sends a very mixed message to our students on what we value and what we care about. We really need to be careful in our faculty conversations what assumptions persist from proctoring and the online cheating conversation.

Inside Higher Ed: The hope, to let you be optimistic at the end, is that lessons that you and many others were shouting throughout the pandemic -- about the answer to more cheating being change up your assignments and your assessments to make them less high-stakes and do all the other things one might do to minimize the instinct to cheat -- were embraced and will be even more fully embraced going forward?

Jankowski: One can hope. I also think there is a larger conversation to be had about if we are in a technology-infused environment, what counts as cheating? So if I share exam questions on a discussion board and somebody picks it up and answers that, is that cheating? Or is that like, how are students examining and thinking about how these different technology platforms play into these things, because I think there was also some instances where you had students very surprised, saying, “I didn’t cheat. It’s not like I went out and found these answers,” or “I didn’t take the exam results out of the teacher’s cabinet and write them down.” And so, we have a very high bar in higher ed as faculty on what that is. But if we’re trying to teach collaborative problem solving, is that an instance of that, where students figure that out? I don’t know. So I think there’s also a conversation on what is cheating in an age of technology.

Discussion With Ereka R. Williams

Inside Higher Ed: What’s your sense of how much learning loss colleges and universities are likely to see, both in their new incoming students, and in those who had already been enrolled at the institutions during the pandemic?

Williams: Most of us in the higher ed space are gearing up for the unknown as much as that can be done. For one reason or another, it’s fair to expect some dips in content, content mastery, some dips in comfort and ease of classes are now all face-to-face on the campus, which, that’s not our case here at Winston-Salem State University. We will be continuing in different delivery formats and things we had to explore and open up for the first time in 2020.

You know, no matter how they’re coming to us or returning to us, I think it’s fair and reasonable for all of us to plan for them to have some dips in some of those right-now, day-one-ready skills and dispositions that historically we’ve expected from a new freshman or a continuing junior, or returning continuing student of any status when they came to us in the fall of the year.

Inside Higher Ed: Assuming more students are walking in with these dips, as you describe them, how effectively equipped do you think colleges and universities collectively are to identify those gaps or setbacks?

Williams: Winston-Salem State, we were an institution founded for those at the margins. This is a historically Black university, and just like the other hundred or so of us that remain, it is in our DNA that we assess and determine what those learners who are showing up to our doors, our campuses, our Zooms, what it is that they need, how they need it, and to stand ready to respond.

So our University College is outfitted with just an incredible team of professionals whose job it is to take experiences like the first-year experience. We take all these different opportunities that we get close to students, if they’re traditional students, close to their families, and to really kind of get a handle on or assess where it is that we are inheriting them. Because our belief, and it’s written in our mission as who we are, it’s in our strategic plan, and it’s codified from a social justice lens, is that we’re here to meet them where they are and to get them where they have to go.

Inside Higher Ed: To the extent you’re able to diagnose where students are, what are the tools and approaches you have at your disposal to help them make up any lost ground if they’ve come in with perhaps greater needs than normal?

Williams: We have increased the number of supports and people in the tutoring, writing center space, and in those areas -- all of those services fall under out what we call UCaLL in University College. So every angle with the student tutoring, and the areas that were already in existence before, we’re going to, we’ve added more folk, we’ve got more manpower, womanpower there. We are also going to spend more time and resources on the social and emotional piece with the counselors, as well as academic counseling.

So we use an early alert system -- most universities do. Ours is called EAB Navigate. And so I’m working right now with our EAB team to make sure that we are codifying a little more aggressive attention to the early alert system. We’ve been using it for a couple of years. We’ve gotten better at it over time, as most people do when they get more experience and training, but we are really going to lean heavily on using and raising flags and alerts sooner, and more often, when the smallest of things pop up on our radars. And so codifying that a little more in the system … We’re going to be working with faculty on that piece, and others in those spaces, in the academic affairs spaces that have access to students academically, to not let them slide off our radar.