Traditional Success Metrics Not Enough

Colleges are judged on traditional measurements of student success, like graduation and retention rates. These metrics can influence their funding levels, which is especially important given the pandemic-induced recession.

But these measurements tend to focus on first-time, full-time students who are seeking degrees. These more traditional students are more likely to attend four-year colleges.

Community colleges serve an increasingly nontraditional population of students, said Shauna Davis, executive director of programs at Achieving the Dream, a membership organization advocating for community college student success. Many are older adults, working, parenting or financially independent. These students often start and stop their education several times. They also are more likely to attend college part-time, which can have a big impact on funding for their institutions.

Community college students also need support for basic needs, like food and housing. Graduation rates provide one view of a college's success, but they ignore how the community college might successfully be supporting students outside the curriculum.

These traditional metrics also don't measure the ways community colleges serve their communities, Davis said.

"These colleges serve a wide range of students who will seek employment in their communities of residence, and their success is essential to the economic impact of their families," she said.

A new series of reports from Ithaka S+R, which offers strategic advice and support services to colleges, examines the traditional metrics used in higher education and explores what other metrics could help form a more holistic view of community colleges' successes.

"These traditional outcome measures in the current measures are designed more so for the four-year college space," said Melissa Blankstein, survey analyst at Ithaka S+R. "There are a robust set of holistic measurements out there, but they’re less standardized. There’s room for more adoptions of holistic measurements."

The report takes stock of the current measurements, including the Integrated Postsecondary Educational Data System and the National Student Clearinghouse Research Center.

Both of these databases collect information on degree completion. This can have implications for funding, as more states move to performance-based funding models that focus on degree attainment, the report states. About one-third of community college revenue comes from state resources.

This can be good for community colleges in some ways, as some states reward colleges for enrolling more Pell Grant recipients or students of color, whom two-year colleges disproportionately serve. But it can also lead colleges to chase easy attainments instead of enrollment, incentivizing community colleges to increase completion of certificates and credentials over student transfer or associate degrees, the report states.

"There certainly are limitations to what performance-based funding has been able to produce," said Christine Wolff-Eisenberg, manager of surveys and research at Ithaka S+R.

If the measurements were different, colleges could focus on students' holistic needs to boost enrollment and retainment.

"The biggest challenges that students are facing right now generally go beyond what happens in the classroom, particularly at community colleges," Wolff-Eisenberg said. "Community colleges right now need to be able to address those kinds of challenges more than ever, and how do you do that without having ways of assessing current needs?"

Blankstein hopes that the report will raise awareness of holistic needs and how to measure them. The report highlights several efforts to measure these issues, such as food and housing insecurity, student engagement, and the need for other supports like transportation.

For example, the Hope Center for College, Community and Justice surveys students about their experiences with food and housing insecurity. The Trellis Company surveys students about their financial wellness, which includes measurements of food and housing insecurity. Some college systems, like California State University, conduct their own surveys. The Center for Community College Student Engagement surveys students each year on their engagement, which reveals how often they use services like tutoring. Generation Hope, a Washington, D.C.-based advocacy organization for student parents, surveys that population about their connections to their institutions and the supports they need.

"These students need every benefit of a holistic approach to student success, and they need their colleges to be able to focus on meeting their needs without the friction between funding models based on metrics that don’t accurately reflect the movement students are making towards achieving their individual goals," Davis said. "It’s important for us all to understand that metrics represent real students, real families, real communities and real opportunities."

In the subsequent reports in the project, Wolff-Eisenberg hopes to identify what is preventing more community colleges from taking using these measurements to determine what their students need. It could be a capacity restraint, as many community colleges lack the resources to adequately invest in research staff, she said. The staff they do have must collect data for the federally required reports first.

But it could also be about perspective, she said.

"For a long time, higher education institutions didn’t see it as their responsibility to provide social services to students. This has changed over the past few years," Wolff-Eisenberg said. "We're interested to see how much of that perspective has persisted … Perhaps there are attitudinal shifts that need to happen."

Viewing all college students as a traditional monolith is harmful in and of itself, Blankstein said. For instance, the federal CARES Act provided funding to colleges during the onset of the COVID-19 pandemic by counting students using the metric of full-time equivalency, but not all students attend college full-time. If the relief funds had used student count for the formula, community colleges would have received more money.

The term "nontraditional student" can also be noninclusive, Blankstein said. She prefers the term "post-traditional," as students who don't fall neatly in the traditional category are now becoming more of the norm.

It's key for institutions to have these data on these students so they can better serve them, said Mamie Voight, vice president of policy research at the Institute for Higher Education Policy.

"Voluntary data initiatives have attempted to fill in the gaps, but what we need is comprehensive data reform that finally moves federal collections beyond what was once considered a 'traditional' student," Voight said.

Higher education should measure student pathways within and across institutions, be inclusive of students regardless of how many credits they are taking, and highlight the barriers that impede student success, such as food and housing insecurity, she said. The data also need to be disaggregated by race, ethnicity and income.

"Without complete data that captures the varied pathways and experiences of today's students, including community college students, policy makers are unable to ensure equitable opportunity and success for their students and cannot identify impediments to that success," Voight said.