Where’s the Wheat?

In days of yore, the cures for most ills came from one of two sources. People either flocked to traditional remedies, many of which did the trick, having evolved over centuries by trial and error, or they resorted to trying the newest exotic and exciting concoctions that were thriving on hearsay. Often, charlatans would capitalize on desperation to peddle useless elixirs to individuals willing to take a chance on anything with a possibility of helping.

As scientific discovery and systematic testing increased, so did the move to evidence-based cures. And today, the quest to resolve many teaching and learning challenges in higher education has seen a similar evolution. But many educators still fall prey to pedagogical snake oil.

With the advent of social media, an uptick in publications on teaching and learning, and the rise in blogging, educators now have many sources that offer ways to address teaching challenges. Much of what is being written highlights new techniques and practices to try, and academic buzzwords abound, as Stephen L. Chew and William J. Cerbin have noted. Unfortunately, however, a lot of wanton speculation and advice still masquerade as valid and reliable guidelines. We can find a number of examples of quackery in education where methods or techniques have been promoted even when common research methods have not shown them to be effective.

Caring educators who want to do the best for their students are constantly looking for ways to develop and improve. Even with busy schedules and packed lives, college and university instructors often want to incorporate a new practice or redesign their course to reflect advancements in the scholarship of teaching and learning. Those are worthy goals. But how can teachers separate the wheat from the chaff?

Distinguish Between the Big Three

First, we need to read between the lines when a new technique is bandied about. Thoughtful higher education instructors want to use techniques that result in better student learning, but not all practices are valid and reliable. Validity, or the extent to what something accurately does what it says it does, is particularly crucial. If a teaching practice is valid, using it should lead to an increase in student learning or associated factors, such as student motivation, that can aid learning. Likewise, reliability, or the extent to which something consistently works, is also important. Active learning, for example, is perhaps one of the most reliable ways to increase learning.

How then do you evaluate whether you should invest the time and energy in trying a new technique—with hopes that the results will most likely be valid and reliable? You should consider three terms, provided by the area of implementation science, that can serve as a useful way to differentiate between the ways a technique has been tested and, thus, can help you determine whether to pursue it in your own classroom. Some techniques have been shown to be efficacious—that, is they work in the research lab. Significant research has suggested, for instance, that the use of spaced practice (spreading out studying) and retrieval practice (repeated testing) have demonstrated efficacy.

Other teaching techniques have been found to be effective—that is, they work in actual classrooms. More recently, many in-classroom studies, including some I’ve participated in, have shown spaced and retrieval practices to be effective, as well. This distinction is extremely important, because an efficacious practice that works under heavily controlled conditions may not be effective in an actual classroom.

A third category of practices may be efficacious and effective but have not been tested. Universal design for learning, or UDL, is a prime example. While UDL practices arose from an understanding of neural pathways in the brain, research on testing the link between the practices, the underlying neuroscience and students’ learning is still ongoing. As stated by the founders of the UDL movement, UDL has research promise. It may be efficacious and effective, but that is still to be shown. Techniques with promise need not be avoided, especially when, like UDL they move us toward inclusive teaching, but we need to be cautious.

Rely on the Process

The phrase “research shows,” featured in a news piece, often provides the hallmark of dependability. But before we blindly trust in it, we need to take a good look at where the research was published. The gold-standard process used by science is peer review. Research is written up for publication, submitted to a scholarly journal and reviewed by two or more experts in the field and an editor. If it passes muster, it sees the light of published day. Then what that research shows can be spread.

Too often, however, the research disseminated has not been published in a scholarly journal. Instead, it may first appear in a book, which rarely gets robust peer review, or in a blog post, which gets no peer review. Sometimes research results will appear in a press release that the news media picks up, again without peer review. Even if you have no training in evaluating the strength of research, you can at least rely on the peer-review process (though even that is not always a surefire solution). Look for where the finding you want to emulate has been published.

Consider These Caveats

Few, if any techniques or strategies will work for every learning situation. It is tempting to believe that a single generic practice—for instance, active learning—will always work. The hard truth is that the answer to any question like “Does doing X help learning?” is: it depends. Anyone who tells you otherwise is trying to sell you something.

Some buzzwords, such as “student engagement,” are very abstract, with many meanings; learning is contextual. Moreover, what works for one class, university, instructor or group of students may not work for others. Even if you read an exciting piece of research, remember that your classroom may not have the same factors as those under which the research was conducted.

Educators also don’t always acknowledge two key elements. Often, a technique or modification may seem to work because of a novelty effect. This new technique dishabituates the student—and perhaps the teacher—because it is different from business as usual. A new visual syllabus. A new assignment. It may seem to motivate students and help them learn better, but that result may occur just because the student pays more attention. The effect may fade as when the innovation becomes passé.

Related is the passion effect. Research has long documented the power of personality in the classroom, as Jonathan Zimmerman details in The Amateur Hour: A History of College Teaching in America. While one need not be an extrovert to be an effective teacher, as Jessamyn Neuhaus observes in Geeky Pedagogy: A Guide for Intellectuals, Introverts and Nerds Who Want to Be Effective Teachers, students are often captivated and enthralled by exuberant personalities. Sometimes that may make them work harder and learn more, but it’s not a given.

Look Before You Leap

You can find a number of sources that offer evidence-informed, efficacious and effective practices. For instance, Aaron S. Richmond, Guy A. Boysen and I lay out in our book, An Evidence-Based Guide to College and University Teaching: Model Teaching Competencies, a set of tested model teaching characteristics that characterize what makes a good college teacher. Similarly, Linda B. Nilson’s Teaching at Its Best: A Research-Based Resource for College Instructors provides a wide array of teaching tips with associated research underlying their utility, and James Lang’s Small Teaching: Everyday Lessons From the Science of Learning showcases manageable and practical pedagogical practices.

If you are looking for quick and accessible summaries of some common findings related to teaching and learning, the St. Andrew’s Episcopal School Center for Teaching and Learning Face the Mind Brain Education Facts (a neuro–myth buster activity card set) and Oregon State University’s Center for Teaching and Learning Neuromyth and Learning Fact Cards will also help you sharpen your critical thinking about pedagogical facts and see how good you are at sifting and winnowing the wheat from the chaff. They will help you get up to speed on current research about the science of teaching and debunk some of the myths.

In sum, keep trying new ideas yet also watching out for spurious findings. Onward to better teaching.