Nudging Doesn't Scale Nationally

Nationalized “nudge” campaigns that shower students with emails and text messages to encourage them to apply for federal financial aid do not budge enrollment rates, as education researchers may have hoped based on the past success of smaller-scale outreach.

A study by economists at five universities, released this month by the National Bureau for Economic Research, suggests that consistently nudging incoming and current college students to submit the Free Application for Federal Student Aid had no effect on college enrollment or financial aid recipient rates. Researchers tested a campaign on two distinct groups of students -- high school seniors who applied to college using the Common Application and college students of all levels (incoming, applied but did not enroll, currently enrolled and dropouts) who applied within an undisclosed large state system, said Kelly Rosinger, an assistant professor of education at Pennsylvania State University and one of the six researchers who authored “Nudging at Scale: Experimental Evidence From FAFSA Completion Campaigns.” Her colleagues were from Brandeis and Brigham Young Universities and the Universities of Pennsylvania and Virginia.

Previous research has shown the success of nudging on a smaller scale from sources familiar to students, like advisers or local community organizations, Rosinger said. But for this study, the researchers tested whether nudging would be effective through state- and national-level organizations with broader reaches, like the Common Application, which is one possible reason the outreach didn’t garner results, Rosinger said.

“When we think about scaling up, working with national and state-level organizations, the messaging has to be more generic than the previous messaging had been,” Rosinger said. “Common Application covers the nation, and students are somewhat familiar with it when applying to college … The students have a weaker connection to Common App.”

The Common Application and the state system sent “experimental” text messages to about 700,000 students over all, about 340,682 high school seniors who had registered with Common Application, a majority of them lower-income and first-generation students, and 350,407 incoming, current or former students who had applied within the state’s higher education system.

All 800,000 students in the study received some type of standard text message with federal financial aid information, but the experimental groups had further communication, like emails, infographics, mailers and varied text message content tailored to students’ identities. For example, if a student was identified as low-income, texts included the benefits of receiving financial aid.

“It didn’t seem to matter how we framed the message or how we sent the message; we weren’t finding differences between them,” Rosinger said.

The experiments produced no substantial results, and Rosinger and her colleagues will likely continue to pursue the possible explanations for students’ unresponsiveness to large-scale nudging, she said. They hypothesize that more widespread nudging does not take into account individual students’ needs and that the messaging is too general to connect with students.

“If they don’t know the messenger, and they’re coming out of the blue, it’s also a matter of trust and skepticism,” said Philip Oreopoulos, a professor of economics at the University of Toronto, who has also done extensive research on nudging strategies. “More and more, as texting and email have been used as efforts to try to market or pitch ideas or thoughts to younger generations, I think they’re becoming more skeptical and disinterested in the messages. It becomes more automatic to tune them out.”

“In contrast, if the effort of this type of information, a reminder campaign, were coming from a friend, the school, a teacher, where you felt like paying more attention to the message, at least at the beginning, the message becomes more salient,” Oreopoulos said.

Another possible explanation for the null results, the study argues, is that information about FAFSA submission is distributed more widely by other sources than in the past, and students don’t need the additional information or assistance these nudges attempted to provide.

The researchers do acknowledge that “complexities associated with [FAFSA] can deter college-ready students from enrolling or succeeding in higher education,” specifically low-income or first-generation college students, whose families are less likely to have experience applying for federal financial aid, but the results show that these students in the Common Application system were generally uninterested in receiving assistance with the process.

Nudging efforts are particularly aimed to provide support that’s absent when disadvantaged students’ parents or high schools are not as involved in the college application process, Oreopoulos said, but text message reminders are much less effective than sitting down with an adult to complete the FAFSA.

“Parents from more advantaged backgrounds are already on top of application deadlines and making sure their child is aware,” Oreopoulos said. “In many ways, this is trying to level the playing field. [But] there are some better ways to level the playing field than others -- having schools work through applications with students is much more like having a parent there to help.”

The “Nudging at Scale” researchers acknowledged this by sending 2,000 of the Common Application sample students text messages offering one-on-one advising through College Possible, a nonprofit organization that provides college admissions coaching for low-income high school students.

Only 11.6 percent of these students even responded to those messages, according to the study, and again, there was no significant difference in FAFSA application or college enrollment results when the researchers compared the one-on-one advising students to those who received more standard text messages about federal financial aid, Rosinger said.

There’s still hope for nudging techniques -- it’s promising that a large organization like the Common Application agreed to assist with the nudging experiment, Oreopoulos said, and the “Nudging at Scale” team remains excited about the effectiveness of the strategies over all, according to Rosinger. Through his own research, Oreopoulos has found that it’s extremely difficult to nudge students to change study habits or other behavioral tendencies, even on a smaller scale, but behavioral economics remains one of the cheapest ways to prompt students into enrolling and completing college, he said.

“I haven’t lost hope,” Oreopoulos said. “I’m cautiously optimistic that there’s still things we can do, but [for] the deeper problems of lack of access and lack of completion, I don’t think we’re going to be able to address those in a meaningful sense just by nudges alone … Maybe more meaningful impact depends on more heavy lifting.”