A Push for Equitable Assessment

A few years ago, the National Institute for Learning Outcomes Assessment noticed that some discussions in higher education were shifting.

The sector was starting to focus more on equity when talking about recruitment of students and teaching practices as the needs of students began to change, said Erick Montenegro, communications coordinator and research analyst for the institute and a doctoral candidate at the University of Illinois at Urbana-Champaign.

But, in the assessment world, not much was moving.

As a Latinx, first-generation learner, Montenegro couldn't see himself reflected in the data collected in assessments. To get the conversation going, the institute in 2017 released its first report on equitable assessment, a model that sees assessments as ways to determine how well students are learning and to identify what teaching practices may need adjustment. For example, rather than promoting difficult testing where most students fail, colleges should instead assess students in myriad ways to see if they have reached specific learning goals.

By 2018, institutions were starting to realize they needed to shift, Montenegro said, but they needed examples of what equitable assessment could look like. To address this, the institute is releasing a series of case studies on the strategies colleges are using to make their assessments more equitable.

"When you validate someone’s learning, you validate them as a person," said Natasha Jankowski, executive director of the institute and a research associate professor at Illinois. "And if we’re not being very mindful of that, we’re not doing the right thing."

The institute has already released three case studies (from Cornell University, Portland State University and Capella University), and expects to release a total of seven. The goal was to highlight varied examples so that colleges didn't approach the work as simply checking off a box, Jankowski said. The institute also wanted to ensure that colleges could no longer cite a lack of examples as an excuse to ignore the issue.

"We can do equitable assessment everywhere -- we just have chosen not to," Jankowski said. "But we can’t afford to not focus on this anymore."

The case study from Portland State shows how it has integrated equitable assessment practices into its university studies program.

"Our case study represents some of our longtime practices and some of our newer ones," said Rowanna Carpenter, director of assessment for the university studies department. "It's not that we’re rethinking our approach, but that we’re deepening our emphasis on equity."

The faculty group that founded the department about 25 years ago chose to focus on learning outcomes, Carpenter said. From the beginning, the department has used practices like having students create portfolios, and now e-portfolios, to demonstrate they met the department's learning goals. Students can include written reports, artwork, podcasts or anything else they think demonstrates their learning. The portfolios are then graded using rubrics.

In this way, students' learning is not judged solely on a high-stakes test, but rather with a curated collection of activities from the students, which can also include evidence of their learning from outside the classroom. The students' curation of the portfolios is also important, as it demonstrates how they themselves understand their learning. The results are disaggregated into subgroups of students so that faculty can pinpoint which students may be underserved.

Because 90 percent of students will take courses in university studies for their general education requirements, it's important for the department to be inclusive of students coming from all different majors and abilities, Carpenter said.

Faculty review the portfolios using rubrics, and they take a sample to determine how the program itself is doing and where they may need to improve to better help students, she said.

The practice is definitely transportable and is used at many other colleges, she said. One of the important pieces is to shift how colleges approach assessment. Her department sees it as a way to bring faculty members together to improve teaching and learning.

Carpenter submitted the case study to the institute in part because she wanted to highlight how the university supports its part-time and adjunct faculty in this endeavor. They are paid to participate in the portfolio reviews and conversations.

"That's an area of equity that isn't always part of the conversation," she said.

Assessment has generally been based on scientific principles and objectivity, according to Gavin Henning, a professor of higher education at New England College and an expert on this issue.

"It goes back to the Enlightenment, when the whole goal was to separate the knower from the nonknower," Henning said. "It forces voices to be silent because we focus on more generalizable research and knowledge, and we focus on what the majority think."

This approach ends up homogenizing student voices, he said, so the voices of underserved students and those in smaller demographic populations, like Native Americans, are lost.

"We have to start thinking about how we can use assessment to stop perpetuating oppression, and how we can use assessment as a tool for equity," Henning said.

Many colleges, for example, assess students' ability to write by having them produce research papers, when there are many ways to write, Jankowski said. If the learning outcome goal is to ensure students have the ability to write or communicate ideas, it would be more equitable to let them demonstrate this in whatever form suits them best. Jankowski added that some specialized skills, like those required of nurses, can't be learned more than one way. But many general skills that colleges hope to teach all students, such as critical thinking or communication, can be expressed in many forms, she said.

She hopes this series and the institute's other reports push institutions to make assessment about students and their learning, not just measurements.

Jankowski is optimistic about this work given that many colleges have reconsidered standardized testing requirements in recent months and instructors have adjusted their assignments and student expectations in light of the coronavirus pandemic.

"I’m optimistic that we will see a continuation into the fall. I’m most worried about the lasting change," she said. "We need to be watching out for that stuff and planning our messaging to get it into the water supply. This opened our eyes, but we must refuse to shut them."