Checking on Vendors

In the competitive world of college admissions, first impressions are crucial. But if your institution outsources its communications with prospective students to a third-party vendor, how would you know if that vendor is pulling its weight? What if inquiries from qualified candidates went unanswered?

One solution, employed by an increasing number of both traditional and online-only institutions, is secret shopping. The method is particularly popular for reviewing admissions procedures, but secret shopping can be used to look at almost any aspect of how universities might interact with students, whether on campus or off and whether managed by the university internally or by third parties.

Secret shopping is a common tactic in the retail and hospitality businesses. Secret shoppers are employed to go into a business and act like a regular customer, perhaps with a particular scenario to test. Unbeknownst to staff, that shopper carefully records details about their experience, which will be fed back to the company’s management.

Maria Jump, assistant vice president of student services at Colorado State University Global Campus, an online-only institution in the CSU System, started using secret shopping last summer and now conducts around 20 “shops” per month. Her focus is on evaluating and improving the student experience, and secret shopping gives her information she “just couldn’t get” from student surveys, she says.

“Secret shopping allows us to focus on one piece of the student experience and control what we’re trying to test,” said Jump. Rather than asking students to reflect on their general experience in retrospect, as usually happens with a survey, secret shopping allows Jump to see how the university is responding to specific situations.

While Jump frequently tests functions that the university runs internally, such as enrollment and advising, she is also increasingly using the service to test functions it outsources. Previously, the institution lacked the ability to “truly assess whether our vendors are upholding our service standards with our students,” said Jump.

Jump identifies possible problems with vendors using responses from student surveys. She can then design shops to figure out if a student’s experience was a one-off or part of a deeper issue. Jump said vendors the university works with, such as online tutoring service Smarthinking from Pearson Education, have been responsive to feedback from shops.

“We want to be transparent with our vendors about what we’re doing,” said Jump. “So we’ll go to them afterwards and say, ‘Hey, this is what we’ve done, and these are our findings,’ and they’re actually very appreciative of that information, because they don’t always get that insight.”

CSU Global works with a company called Campus Feedback to organize its secret shops. Campus Feedback is part of a larger organization called Goodwin Hospitality -- a national provider of secret shopping services for the hospitality industry. Jump said Campus Feedback was one of the only companies she found that offers secret shopping for higher education, although companies such as Render Experiences and Demand Engine offer similar services for campus visits and prospective student interactions, respectively.

Kurt Eddins, senior vice president of business development for Goodwin, said Campus Feedback has worked with around 100 institutions, including Brown University, Southern New Hampshire University and Hunter College in the City University of New York System. The company says it can help colleges look at their student experience “from start to finish.”

“Every interaction with a student or prospective student, or family member or alumnus -- those are critical interactions in this competitive environment,” said Eddins. Admissions is the most popular area for Campus Feedback in higher education, but shops can be organized for almost anything, he said.

Often institutions want feedback from their own students, whom Campus Feedback recruits to serve as secret shoppers. The institution never knows the identity of the secret shopper, and the shoppers are compensated for their time. Shops can include making phone calls or sending emails to test a service, or reviewing on-campus experiences such as dining halls, bookstores and athletic events.

Increasingly, institutions want to use secret shops not only to look at their own services, but for their vendors, said Eddins. “So many of these schools have so many vendor relationships, and they assume everything is going as planned, but that is not always the case,” he said.

In recent years, colleges increasingly have outsourced key university functions. Though there are caps on how much universities can outsource from the academic side of a program, Republican policy makers in favor of deregulation could relax those rules. While outsourcing can make financial sense for institutions, and vendors can offer expertise colleges may not have, the rise of secret shopping of vendors highlights how challenging it can be for institutions to monitor whether their partners are operating as they would expect.

“I think if more institutional leaders knew they could secret shop their vendors, they would jump on it,” said Eddins. Secret shopping used to have something of a bad reputation, said Eddins. The perception that it’s only used to look for problems, he said, has slowly become seen more as a constructive tool for assessment.

Alison Vujnovic, director of enrollment at Oral Roberts University, a private Christian university in Oklahoma, has been working with Campus Feedback for three years. Vujnovic said that she has been using shops to really “dig deep” and get an “inside look” at her institution's admission process from the perspective of a prospective student. In particular, Vujnovic regularly sends secret shoppers to open days on campus, where they evaluate everything from the car-parking instructions to food in the cafeteria and professor-student engagement in sample lectures.

Vujnovic said that she is up front about her methods with faculty and staff. She said that most people find the feedback helpful, even if it is negative. If someone gets a poor review, Vujnovic said, she will work with that person to ensure they get necessary training. Vujnovic said she has not used secret shopping to evaluate vendors, but that it is an intriguing idea. “We entrust so much in our partners,” said Vujnovic.

Perhaps part of the poor reputation of secret shopping in higher education is the practice of secretly scoping out the competition. Rival universities often want to look at each other’s practices to see how they compare, said Eddins, and this is a service Campus Feedback used to offer. But recently the company decided to discontinue the service. “It’s something we grappled with,” he said, “as we grew, we began to feel uncomfortable with it.” One reason is that competitive secret shopping could present a conflict of interest for the company, Eddins said, for instance, if it was seeking to build a good relationship with one client and reported information on them to another client without their knowledge. “We want to be an unbiased third party and provide a service for these schools to evaluate themselves,” he said.

Eddins said that requests to evaluate competitors are common because “it’s sometimes easier to shine a light on the competition than look in the mirror.” In the future, however, the company is thinking of ways it can provide valuable benchmarking data --  aggregated and anonymized -- to institutions so they can see how they compare to their peers. “The best clients for us are those that are really willing to engage and open up the hood to see what’s going on,” said Eddins.