Can You Verify That?

Through the last year, in which five colleges and universities have admitted submitting false information to U.S. News & World Report, the magazine has maintained that there is no evidence of a widespread problem, or a need for new ways to assure the accuracy of its data.

But U.S. News is considering a plan in which colleges would be required to have a senior official -- someone outside of enrollment  management and serving at the top rungs of administration -- sign a statement vouching for the accuracy of the information being submitted.

In an interview, Brian Kelly, editor and chief content officer at U.S. News, compared the idea to the Sarbanes-Oxley law, adopted by Congress in the wake of various business scandals, which among other things requires CEOs to certify that they stand behind financial reports being filed. The idea is to create incentives for those at the top of organizations to push for honesty in the materials produced about their organizations. "The integrity of data is important to everybody," Kelly said.

Kelly said that presidents or other senior leaders of colleges and universities could devise their own systems for verifying the accuracy of data, but that they would need to personally certify the truthfulness of what was being provided. He said that there was no need for U.S. News to come up with an exact method for verification. "We're not the accreditor. We're not the federal government. We don't want to be a regulator. We're journalists," said Kelly. "The burden is on these schools to be accurate. It's their reputations. It's their customers."

For U.S. News, even considering the need to add a level of verification would represent a major change. The magazine's editors have insisted until now that there is no need to fear a larger number of institutions might be submitting inaccurate data, or to change any procedures. And Kelly insisted that he was not convinced that there is a broader problem -- only that it's appropriate to consider whether more verification is needed. He said that the five colleges in the last year are a small fraction of higher education. Asked if he considered it likely that more cases would be revealed, he said, "Sure," but he said that doesn't mean he's expecting a lot.

"These are institutions that teach ethics. I find it incredible to contemplate that institutions based on ethical behavior would be doing this," he said.

Kelly also maintained that he viewed the various reports of the last year as "a positive" for the rankings. "People are being responsive to the importance of accurate data," he said.

The floating of the idea of a new level of verification comes amid evidence that many people assume that fudging of data is widespread. Inside Higher Ed's survey of admissions directors last year found that 91 percent believed that some institutions besides those that had been identified at the time had reported false scores or other data.

When U.S. News last month issued a statement about the inaccurate data (this was after four colleges had come forward about falsehoods but before Bucknell University did so), and said that there was no reason to believe the problem was widespread, commenters on this site were doubtful.

One wrote of the magazine's statement: "Unbelievable. Just unbelievable, in the truest sense of that word. Give us a break, USNWR. You have no reason to believe ... ?? What more evidence do you need? You never knew that the cheaters were cheating, until it came to light. What makes you think you would know if someone else was doing so now?"

One of the more outspoken critics of the U.S. News rankings -- Lloyd Thacker, founder of the Education Conservancy -- said he didn't think a verification system would make a major difference in the value of the ratings. Via e-mail, Thacker said: "This sounds like either a business strategy for a battered industry to acquire some semblance of legitimacy, or an educational opportunity for college presidents to think critically about commercial rankings and move beyond such influence. It's not both."

Jerome A. Lucido, executive director of the Center for Enrollment Research, Policy and Practice at the University of Southern California, is also a rankings critic, but he had mixed reactions to the possibility of adding a requirement that senior officials verify information that colleges send in.

"Asking an institutional leader to verify and sign off puts the reporting in a visible place," he said, and that would "appear to be an improvement" because several of the colleges that turned in false data said that one person on the campus had been responsible. Further, the requirement that a president or another senior official sign off would "place visible responsibility" with that person, and the idea "may be worth it just for that."

But Lucido said he wasn't sure such a change would make the data more accurate. He noted that many campuses have groups of officials "who put together their submissions to reach the best possible result," and that "cheating is only one way to manipulate the numbers." As long as colleges focus a lot of energy on rankings, he said, many administrators will continue to think they are justified in doing what they need to stay ahead of the competition.

He said that the "best hope" for the result of a requirement that a senior official verify the accuracy of the data would be that more colleges choose "not to report at all."