Fibbing for Rankings

The University of Missouri at Kansas City gave the Princeton Review false information designed to inflate the rankings of its business school, which was under pressure from its major donor to keep the ratings up, according to an outside audit released Friday.

The audit -- by PricewaterhouseCoopers -- described the process by which business school officials came up with creative reasons to provide data that many at the school believed to be false, and that the audit found to be false. In one case, for example, the university created a wish list of clubs that it might support to promote entrepreneurial students. The university then reported that its wish list was reality and that it had all of those clubs, which in fact did not exist.

Another part of the audit found that an article published in The Journal of Product Innovation Management -- an article that ranked the university's business school as the top institution in the world in the field of innovation management -- did not violate professional norms. However, the audit also found that the journal was unaware when accepting the article that it was written by scholars with ties to the university.

UMKC issued a news release Friday that reads: "Independent review upholds No. 1 research ranking."

But the audit also confirmed many of the findings of an August article in The Kansas City Star that found "a pattern of exaggerations and misstatements" by the business school. At the time, the university disputed the Star's report, but Missouri governor Jay Nixon requested an investigation, and that request led to the report issued Friday.

'By All Means Necessary'

PricewaterhouseCoopers officials had access to senior UMKC officials (including some who left positions they had held in the period covered by the audit) and to relevant e-mail messages. The e-mail revealed a focus on finding ways to do well in the rankings in order to keep happy the business school's largest donor (of $32 million), for whom the school, the Henry W. Bloch School of Management, is named.

An e-mail from the then dean to colleagues said, for example: "Henry Bloch gets very upset when our rankings go down. We must do everything we can to increase it when we can by all means necessary.”

The audit then describes some of the things UMKC did to rank high in the Princeton Review's evaluation of business schools' (undergraduate and graduate) entrepreneurial programs.

For example, in answering a question about how many students are enrolled in an entrepreneurship program, the university started counting anyone who was taking a class in entrepreneurship. Not surprisingly, the numbers jumped. For example, UMKC reported that undergraduate enrollment in entrepreneurship programs increased in a year (the year in which the university changed how it was filling out the form) from 99 to 438. A dean told the auditors that he knew that figure "isn't right."

Another change UMKC made helped it inflate answers on another Princeton Review question: about what percentage of students launch a business while enrolled. The university, the audit found, started using primarily data from its e-scholar program (a certificate program for entrepreneurs in which they must develop a business plan). The e-scholar program students are not degree students or enrolled in the university, but officials said they believed it was legitimate to use this group for reporting, even though the Princeton Review ranks degree programs. Since all of the e-scholar students must create business plans, the proportion of undergraduates reported as launching a business increased from 44 percent to 100 percent from 2010 to 2011.

And then there was the question on clubs. The Princeton Review asks: “How many officially recognized clubs/organizations do you offer that are specifically for entrepreneurship students?”

The answers in 2009 were three each for undergraduates and graduate students, and in 2010 were four each. In 2011 the figure jumped to 29 for graduate students and 28 for undergraduates.

Here's how the number of clubs "grew," according to the audit. A business school official asked a colleague to put together a wish list of clubs that might show an entrepreneurial focus at the university. A second official "then instructed a UMKC graduate student to populate these clubs onto the university’s webpage." UMKC "used the clubs' existence on the university’s webpage as the only proof the club existed." Officials believe "these additional 20-plus clubs never actually existed at UMKC." Since the Star article, the number of clubs being reported is down to five each for graduate students and undergrads.

The PricewaterhouseCoopers report says the Princeton Review does not review the accuracy of information submitted to it by colleges and universities and so did not do any independent analysis of UMKC data. The audit also said it was not clear that any of the false information would affect the business school's overall ranking.

But on Sunday night, Robert Franek, senior vice president and publisher of the Princeton Review, said in an email to Inside Higher Ed that Princeton Review would be removing UMKC from the lists of best colleges and business schools for entrepreneurial programs.

“At The Princeton Review, for the past 34 years we have provided accurate and timely information to students and parents to help them make decisions about colleges and graduate schools. We were extremely disappointed to learn that  the University of Missouri-Kansas City falsified data about the school per a report from PricewaterhouseCoopers on January 30. As a result of this new information, we are removing the University of Missouri-Kansas City from our 2014 ranking lists of the best college and business school entrepreneurial programs," said a statement Franek released. "Schools earn a spot on our entrepreneurship ranking through school-reported data. Every school signs an affidavit to ensure their information is accurate. We take these affidavits and this news very seriously.”

Questions on a Journal Article

Another major part of the audit was a look at the journal article published in The Journal of Product Innovation Management.

On this question, the audit found that the article was based on data analysis and that no shortcomings could be found in it. But the article has been questioned from the time it was published. The original Star article quoted a professor (anonymously, because he feared speaking out) as saying that “We all knew that this was bullshit. We knew that UMKC was not better than MIT and Stanford.”

While the audit didn't question the article's findings, it did note concerns about it. The authors who asserted that UMKC was tops in the world in innovative management did not disclose to the journal that they were both visiting scholars at the university and knew some of the players. Because the article was based on data (number of articles written in journals of various influence, etc.), the journal's editor said that the article's findings still stood. However, he said he wished he had known about the authors' ties to the institution they praised.

The authors are two scholars from China. They gave a letter to the auditor in which they said that there was no need to identify their UMKC connections because the "double-blind" peer review process -- in which they don't know who reviews their work, and the reviewers don't know the author -- prevented conflict of interest. The audit, however, found that at the journal in question "papers are solely reviewed by the editor and not subject to the typical double-blind review of other research papers."

While UMKC issued a statement saying that the audit affirmed the top ranking for its business school, the statement also said that the business school "has already made changes to the way the data was collected and reported this year." Further, the business school "has appointed a special faculty committee to oversee the processes involved in any rankings submissions going forward."