Trial and Error: Scheduling Assessments

In this new feature, we're trying to create a space to discuss institutional experiments with digital technologies -- what worked, what didn't, and what other colleges can learn from the attempts. (The Online Learning Consortium's Effective Practices program has contributed initial ideas.) Send your nominations for future columns to digital.learning@insidehighered.com.

***

The Institution: Western Governors University

The Problem: Eight years ago, the nonprofit institution, which is fully online and provides a competency-based education to adult students was increasingly struggling to schedule proctors for the many assessments its geographically scattered students take. A staff of 15 worked full time to process 4,000-5,000 monthly requests for proctored assessments by setting up appointments (typically requiring 7-10 days' notice) at one of 2,000 affiliated testing centers nationally. As enrollments grew by 20 percent a year, the university struggled to keep up with demand.

The Goal: Create more scalable ways of scheduling the proctoring of assessments to meet students' needs.

The Experiment: WGU pursued two tracks, over a period of years. First it created a system for online proctoring, using initially one and now two providers (ProctorU and Examity, at its different schools). Student requests are routed to the proctoring companies and are usually fulfilled immediately.

The vast majority of the current load of 30,000 assessments a month are done online; Adel Lelo, senior manager of assessment solutions at the university, said he thought it would be a "no brainer" for all students to jump at the chance to "sit in their kitchen in their pajamas in 10 minutes rather than schedule days in advance and drive to the nearest testing center." But 6 percent of students continue to prefer in-person proctoring for their assessments, Lelo said, because they don’t have good technology at home, “the house is messy and they don’t want the proctor to see it,” or they “just want to get away – for some parents, it’s the only alone time they get.”

Those students still take several thousand assessments a month, and for them the university built an automated scheduling system that sends students’ three suggested dates and times directly to a nearby proctoring center, which can accept one of them, reject all of them, or suggest an alternative. That goes back to the student, who responds – all without any involvement (unless there’s a technical problem) from WGU employees. One staff member now coordinates relationships with the testing centers, down from 15.

While the system cost roughly $40,000 to build – more than many other institutions might be willing or able to spend -- the university has been able to reallocate (to other student services) the time of most of the 15 employees who scheduled these appointments previously.

What Worked (and Why): Where scheduling assessments used to take between 7 and 10 days, the in-person appointments are now scheduled in an average of 16 hours, and the electronic appointments are virtually immediate.

What Didn’t (and Why): The university has pivoted at various points. It originally used just one proctoring vendor, but when it had a problem and dropped that vendor, it took nearly a year to switch fully to a new one. “We told ourselves from them on that we would always need to have two,” said Lelo. “Plus competition is good.”

When WGU switched to virtual assessments, it expected to be able to close down its relationships with the 2,000 testing centers, saving headaches. But it learned that some students continued to want an in-person option.

Next Steps: The university plans to continue to integrate its various assessment systems with those of the proctoring companies to, for instance, automate the posting of students’ scores on assessments. Numerous colleges have asked Western Governors for advice on their own assessment systems, and the university has prepared a webinar on the topic.

For more information: Adel Lelo, WGU