My Harvard Embed

As the Inaugural Visiting Fellow at HarvardX, my mission over the six months while “embedded” with the HX team involves exploring how to maximize the effectiveness of assessments and other student assignments within Harvard’s growing catalog of Massive Open Online Courses (MOOCs).

While two decades in the professional testing industry provided grounding in both the science and artistry of test design, a more recent experience – last year’s Degree of Freedom One Year BA project – taught me the important role testing plays in teaching-and-learning (not just the post-learning measurement) processes.

That “One Year BA,” which involved trying to cram four years of college into twelve months using only MOOCs and other forms of free learning, provided a student’s eye view of what it takes to get the most out of a course one is taking alongside thousands of other students.  And while courses varied widely in terms of level of demand, those which asked me to put my learning to work by providing challenging assessments or frequent writing assignments stood out as the most engaging (and meaningful) learning experiences.

Perhaps the term “assessment” is too loaded, given that it triggers fear among students tested and judged throughout their school careers, not to mention agita for those who had to grade hundreds of exams and writing assignments.  Which is why I’m gravitating towards categorizing everything the student is asked to do while taking a MOOC (or any other type of learning experience) as “Active Learning Components,” in contrast to components like lectures and reading which involve absorbing – but not necessarily using – information.

Not that lectures or reading should be considered in any way passive.  For, as I also learned last year, it is only by paying close attention, not multi-tasking, and taking notes as diligently as you would if a professor was standing three feet from you can you get the most from video lectures.  Similarly, completing reading assignments and generally treating a MOOC with the same seriousness you would put into a paid-for residential college class is key to getting the most out of the experience.

But what separates a MOOC from lectures on the same subject downloaded from iTunes U are those moments when students are asked to put their learning to work, which explains why I got so much more out of those massive open courses that didn’t just give me a certificate for showing up.

Both conversations (and my own writing) about assessment within MOOCs have tended to focus on methodology: multiple-choice quizzes vs. peer graded essays, encouraged vs. enforced contribution to discussion forums, etc.  But I would suggest that course developers start with a different set of questions, long before an assessment methodology is chosen or a single test question written.  These questions would include:

• What is the overall goal for the course?  Do you want students to master a body of knowledge (like the Python programming language or the material science involved with frying an egg)?  Transform the way they engage with familiar material (like the works of Homer or the statue of John Harvard)?   Trigger conversation about important matters (like ethics)?  Once a goal is defined, it then becomes easier to determine which types of active learning components can move students towards that goal.
• How can we make our assignment as interesting as every other component of a course?  I can’t tell you how many times last year the symphony of a well-honed lecture was followed by a kazoo recital of a handful of multiple-choice questions where every correct answer was D (“All of the above”).  Yet when test questions were thoughtful and challenging, when writing assignments provided the chance to reflect on what I had learned, when new types of assignments (like crowdsourced projects) created active learning experiences that had no classroom equivalent, those classes stood out (and continue to stand out) from the pack. 
• How can we create assessments that are just as high quality as every other component of a MOOC?  A century of testing science combined with the craftsmanship of item writing has given us multiple-choice question that can be used to measure higher-order thinking skills or engage students in complex interactions with the real world.  And unlike video lectures (which can be expensive to produce), quality assessment takes time to craft, but costs almost nothing to automate. 

MOOCs have already challenged many pre-conceptions of what can and cannot be taught online.  Which makes assessment (or any other form of active learning) the next frontier where organizations like HarvardX can create and innovate in ways that will inspire anyone involved with developing meaningful technology-driven educational experiences (massive or otherwise).

Jonathan Haber is Chief Learner at Degree of Freedom. His book, MOOCs: The Essential Guide, will be published by MIT Press in October. (Note: A version of this blog first appeared on edX.org.)