Equitable Exams During COVID-19

In March, shelter-in-place mandates changed the face of education. Spring breaks were extended as instructors worked to move courses online. Higher education publications like Inside Higher Ed have released countless articles analyzing, predicting and commenting on how the unprecedented move would impact students, instructors and educational institutions.

Now the dust has settled, and we are in May. Most of us have adjusted to the new normal of online instruction. We are adapting to the more frequent use of learning management systems and are comfortable with setting virtual class meetings, and classrooms across America sit empty as students engage from home. But as the spring semester draws to a close, instructors and students will face a final challenge: producing, administering and scoring final exams.

The equitability of online learning was called into question in March when institutions ceased classroom instruction. Educational institutions had to face the disparity of technological access needed to transition students to an online environment. Though recent data from the Pew Research Center show that 73 percent of adults have home broadband internet, the disaggregated data by household income paints a different picture. Fifty-six percent of adults in households earning below $30,000 report having internet. Students with access to technology may be facing other barriers, such as home distractions or the needs to care for family members. To mitigate some of the inequity, some institutions moved to pass/fail rather than letter grades, offering more flexibility to students.

Now, institutions must consider equitable final exams. Even under normal circumstances, instructors must consider the purpose of an exam. Is it formative or summative? Are we testing for skill acquisition or memorization of knowledge? What is the purpose of the time limit? What is an acceptable deadline? As final exams draw near, instructors must consider how to create, administer and score final exams that are fair and equitable during this pivotal time.

As community college instructors and co-coordinators of our department’s assessment committee, we have considered equitable assessments (such as quizzes, midterm and final exams, and informal assessments) the cornerstone of our work. In light of the sudden changes to higher education, here are several considerations for institutions and individual instructors to keep in mind as they prepare for the end of the semester.

Time Limits and Deadlines

Final exams for face-to-face courses usually have date and time restrictions. In such courses, students take an exam all at once during class time, there are time limits and absent students are often required to present reasonable excuses before being given makeup exams. But time limits and deadlines may create additional barriers and stress for students who are struggling to balance the changes caused by COVID-19.

We suggest you consider the following instead:

• Adjust time limits to account for the technology learning curve. Some students may not be comfortable taking an exam on a computer. You should adjust traditional time limits to account for the learning curve.
• Consider getting rid of forced completion. Some students may be dealing with a number of distractions at home -- children, sharing space with family and so on. Forced completion doesn’t take into account interruptions they may experience. Giving students the opportunity to save their exams and return later may relieve the burden of such home distractions and interruptions.
• Provide students several days to complete an exam. Students’ schedules may be inconsistent and heavily impacted by varying responsibilities. For example, some students may have children or siblings at home whom they need to share a computer with. Others may be responsible for taking care of elderly family members. If you are a community college instructor, some of your students may be working to balance school and work. Providing several days to complete an exam will give students the flexibility to select a day and time that works best for them.
• Be flexible with deadlines. Hard deadlines are as out of style as hugs and kisses during the COVID-19 pandemic. Allowing students room to negotiate deadlines and gain some extra time may help relieve their stress.

Access to Exams

One of the biggest challenges that educational institutions faced when they made the quick move to online instruction was that not all students had access to a computer and internet. While many internet providers and companies worked to meet students’ technological needs, barriers have remained. For instance, some students only have tablets, and others may need to share technology with their spouses and/or children. It’s important to consider how students are accessing exams and provide alternatives for students who can’t meet the technological need.

In our course, some students don’t have access to a computer but have a smartphone. They have been able to complete discussion boards and assignments through the learning management system’s app on their phone. However, it would be an added barrier to request that they complete their exam on a phone. Alternatively, they have been given the option to handwrite their exam and send a photo through text, like a modern-day fax.

Also, learning management systems were not all made equal. Some have more user-friendly interfaces than others, and how an exam looks on the system will influence student success rates. Whether the impact is statistically significant is up for research and debate, but you should take into account what your students will be seeing as you design your exam. Be sure that the text of your exam questions is large enough for a student to read with ease on a screen. Reconsider lengthy and complex instructions and opt for simpler, easy-to-understand directions.

Additionally, typing may be difficult for some students. Consider offering alternatives. Smartphones have impressive capabilities. For short answer or essay exams, consider the option of handwritten answers. Students can take a photo of what they’ve written and send it to you with their phones.

Depending on your subject area, you may confront additional challenges. If you’re teaching reading comprehension in developmental education or English as a second language, you may want to consider how students without a computer can access passage-dependent exams. It’s an additional barrier for students to flip between a passage and comprehension questions unless the learning management system allows the passage and questions to be displayed side by side. These are all things to consider as you design your exams.

Types of Exam Items

Consider getting creative with the types of questions you use in your exam. Many instructors might be tempted to create multiple-choice items to ease grading time and provide a measure of standardization to student knowledge. However, considering the unprecedented changes in education, now may be a good time to incorporate different types of exam approaches. Multiple-choice items are great for measuring student retention of facts and can guide students to focus on the topics, but short-answer questions and essays give students an opportunity to showcase their critical thinking and problem-solving skills.

Also keep in mind that the type of assessments, particularly the final exam, proctored during a pandemic may have very different goals than ones given during traditional face-to-face instruction. Traditionally, instructors might require students to exemplify complete mastery of skills. However, given the stress of a pandemic and sudden changes to their academic life, demonstration of complete mastery may not be necessary. Other creative alternatives to assessing student learning may also be beneficial at this time, such as final projects or meaningful written reflections.

If you teach a subject area where multiple-choice questions provide the best means of assessment, consider the following:

• Reduce the number of choices. Typically, instructors will provide four, five or even more choices for each multiple-choice question, but research increasingly shows that three choices work just as well for determining the extent of students’ knowledge, with no decrease to the validity or the reliability of the item. What that means is that the item is still just as rigorous for your students while decreasing the work for you as an instructor. This does the additional job of reducing the amount of time students need to answer individual items while still accurately measuring their knowledge of course material.
• Avoid “all of the above” and “none of the above.” Both these choices reduce the ability of students to differentiate well among the available test choices while providing minimal benefit. “None of the above” is particularly notorious for luring both high- and low-performing students into choosing an incorrect answer. That makes it less reliable as a measurement instrument.

Item Analysis

Item analysis is a process by which you systematically examine student responses to particular parts of an assessment to evaluate their quality as a metric for outcomes. Stated simply, it’s looking at your exam questions and checking that they’re doing what you want them to do. This process is worthwhile at all times, but it is especially valuable now as exams are being rewritten for the online teaching environment. Evaluation of how well students performed may help you determine which questions aren’t serving their purpose and should be re-evaluated.

In one of our reading courses, we realized that a particular type of test question that asked students to define vocabulary, contextualize it and categorize its part of speech all in the same section was doing a substandard job of determining who understood the material and who did not. (That’s called “discrimination” in assessment parlance). We used this data to separate the activities into separate items, which ended up better demonstrating student understanding of the material.

Blackboard has a tool that allows you to run an item analysis from inside a test canvas. This tool approaches the issue from a purely mathematical standpoint, and you’ll have to do a little legwork to interpret the data on your own. If you’d like to approach this task manually, a TurnItIn blog post shows the kinds of questions to ask yourself as you design and review exams.

Rubrics

In the absence of face-to-face guidance, clear criteria for expectations become more important. A good rubric makes those expectations explicit and describes what learning looks like. Once you’ve decided to make a rubric, you’ll need to choose either an analytical rubric (measuring criteria separately) or a holistic rubric (measuring all criteria together as part of a whole).

To use an English class as an example, an analytic rubric might assign points separately to the grammar, style and cohesion of an essay for a total of 100 points. A holistic rubric would instead consider the grammar, style and cohesion as simply parts of the whole and assign a letter grade for the overall strength of the essay. While the former is a good pedagogical choice toward the beginning of the semester, you might consider giving students a break from reading detailed rubrics and yourself a break from filling them out by transitioning to a holistic rubric as the semester progresses. This ASCD article on how to create and use rubrics provides a detailed look at the construction of a good rubric.

Final exams are a large part of higher education and what has been the traditional student experience. With COVID-19, campuses sit empty while instructors and students navigate instruction during a pandemic. As you approach final exams, it’s vital to reconsider time limits, deadlines, student access, the types of exams created and how to evaluate exams. Such considerations provide a space for flexibility, understanding and, ultimately, student achievement.