Teaching to the (Right) Test

“Teaching to the test.” Surely, in any world‑class educational institution this must be a bad idea. From professors to deans to education experts to parents, it is commonly denounced as a practice signaling a failing strategy that is on the wrong track.

Opposition to practices that emphasize performance on high-stakes tests commonly starts with K-12 education, and the criticisms may be most passionately articulated there. In last year’s State of the Union speech, President Obama seemed to concur: “Let's offer schools a deal. Give them the resources to keep good teachers on the job, and reward the best ones. In return, grant schools flexibility: To teach with creativity and passion; to stop teaching to the test....” But this sentiment that begins in the K-12 arena often carries over to higher education situations and can undermine thoughtful implementation of assessment in curriculum design and effective teaching.

The rationales given in opposition to curriculums designed around testing are numerous. Testing crushes creativity, narrows the curriculum, doesn’t focus on what students actually learned, and leads to unfair evaluations of students, with results that do not predict useful things. Purported outcomes of teaching to the test include learning that is short-term and not generalizable and the inability to make valid inferences about content mastery.

We can readily summon facts from lessons drilled into us decades ago when we were in grade school: Columbus sailed on the Niña, the Pinta, and the Santa Maria. The Civil War ended in 1865. Gregor Mendel was a monk. But does knowing these facts reflect mastery of important concepts and skills? Of course not.

Bad tests are not helpful. And teaching to bad tests is counterproductive. These facts, however, do not compel a conclusion that testing itself is bad or that teaching to tests is bad. This erroneous logic, unfortunately, has hijacked the national conversation about educational reform, and hindered innovation in higher education.

To the contrary, teaching to the test is good... if it's the right test. As we begin a new academic year, whether we’re teaching a course we have taught many times before or developing a new one, the time is right to reflect on testing and its appropriate role in our own teaching.

Let's erase the board and start fresh. Are there generally accepted aspects of good curriculum design? Yes. Step 1 is to decide what we want our students to learn: the learning goals. Step 2 is to devise ways to determine whether (and how well) our students are accomplishing these goals. And Step 3 is to design our curriculums and teaching strategies around gaining mastery of the learning goals.

Establish a goal. Figure out how to evaluate performance. Work toward the goal. For goal‑oriented endeavors -‑ the design of medical treatments, the quest for athletic performance, the creation of culinary delicacies -- this three-step process works.

Educational innovation is an iterative process. Preparing our lectures and designing our curriculums aren’t the end; they’re closer to the beginning, in fact. We must demand evidence that education -- in general and in each course we offer -- is “working.” When we have articulated the learning goals and designed tools for assessing their attainment, we are in a position to get that evidence.

In assessing our students, we can determine how well our curriculums and teaching strategies are helping them meet the learning goals. Importantly, we can also use the evidence from the assessments to modify and improve the efficacy of our lesson plans and teaching strategies. This evidence from assessments guides us toward the more effective methods, and helps us revise the less effective ones.

Have you ever waited until a day or two before a midterm to begin writing the exam? We have done this. And we have, only then, realized that for some topics we were unable to ask the questions we wanted; we had not covered the material in ways that adequately prepared our students to write the complete and precise responses required. Or we struggled to remember the precise details we included (and excluded) in a lecture many weeks earlier.

When we develop our tests and the curriculum that supports those tests in concert, the process can lead to fast and significant improvements. Only to the extent, however, that our tests are appropriate and powerful for assessing progress toward the learning goals. In this domain, fortunately, there has been tremendous progress since the days of Niña, Pinta and Santa Maria regurgitation.

"Good test!" Your students may never exclaim this. But there are indeed good tests and bad tests. What are the features of a good test? Validity and reliability are at the top of the list. Validity measures how well items correspond to learning goals. Reliability reflects the consistency with which a test measures what it is supposed to be measuring when given to different groups of students, in different environments.

Should our students be able to answer only questions that we have written specifically for our “version” of a class? What about assessment items from other sources? Concept inventories relating to the core ideas in a discipline, for example, have been developed and published by content experts in many fields. Textbooks and other resources are available to instructors (and students), too. Can these be useful?

Test questions and other assessment tools -- regardless of their source -- when written to clearly defined learning goals, are helpful. Perceptions of misalignment and unfairness arise when the learning goals are not clear to both the test creators and the teachers. A disastrous reality is that most teachers in higher education today receive little or no training in assessment. This mirrors the situation among K-12 teachers. A 2012 report by the National Council on Teacher Quality revealed that only “3 percent of teacher preparation programs adequately build teacher's skills in the crucial area of student assessment.” We all can use help and we all can improve.

Ideally, test design begins with content expertise and is supplemented by a deep understanding of student misconceptions. Questions must then be revised and improved in response to quantitative analysis that reveals an item’s power to discriminate between students with subtly differing degrees of mastery of ideas. Tests can and should be constructed to maximize the information provided about student performance across the entire range of the established learning goals.

Put another way, as high‑tech mavericks have surged forward with innovations changing nearly every aspect of our daily lives and our interactions with information, education researchers (who also happen to be parents and teachers) haven't been out at recess. Advances in assessment design and analysis have been equally revolutionary.

A common objection to standardized testing is that it reduces the recognition and nurturing of creativity. Neither testing nor teaching to the test are necessarily in opposition to creativity. Creativity can -- and should be -- one of our learning goals. As such, it should be carefully defined and articulated. Great strides have been taken toward identifying and enhancing, as well as assessing, components of creativity, including curiosity, motivation to generate original ideas, and the ability to tolerate ambiguity, integrate opposites, and examine assertions.

Planning educational reform -- at all levels, including the reform of our own course offerings -- should include reflection and debate about learning goals and innovations in teaching. Our focus must also, however, be on the design of reliable assessment methods closely aligned with the learning goals, and evaluations of the learning gains associated with any teaching innovations.

Test results can be an unpleasant wake‑up call for students as well as instructors. Our students may not know what we think they know. They may not have learned what we thought they should and would learn. But worse than getting a wake‑up call is not getting one. Good tests provide valuable information and they can and should guide us in designing our courses and our curriculums. When learning goals are clearly defined and reliable assessments are aligned with them, "teaching to the test" is not only good, it is exactly what we should be doing.