Rankings Rancor at Clemson

"Wow, think her bosses at Clemson knew she was giving that talk?" one audience member said to another as they walked out of Catherine E. Watt's presentation about the South Carolina university's approach to the U.S. News rankings Tuesday at the Association for Institutional Research.

Judging from the combustible reaction, it's safe to say not.

Clemson officials lashed out at Watt in a prepared statement Wednesday after Inside Higher Ed reported on her statements at the researchers' annual forum in Atlanta. Watt, who until 2006 headed Clemson's institutional research office and now leads a research center there, made a presentation that was nominally meant to laud her university's effective strategic planning and project management efforts.

But the "project" whose success she described -- Clemson's single-minded pursuit of its widely publicized goal of becoming a top 20 public research university -- resulted in an unusually frank airing of how what Watt called "manipulation" of such measures as class size and faculty salaries has helped Clemson soar up U.S. News & World Report's ranking of public research universities. In her speech (a summary of which can be found here), Watts asserted -- among other things -- that Clemson officials, with rising in the rankings as their central goal, had systematically lowered class sizes below the U.S. News threshold of 20 while increasing class sizes where the additional students wouldn't hurt its standing in the rankings; and that they had regularly given low scores on the rankings' "reputational" survey to other colleges and universities in order to make Clemson look better.

In a written statement and in an interview late Wednesday, Cathy Sams, Clemson's chief public affairs officer, took strong exception to Watt's presentation -- although she mostly offered alternative explanations for the reduced class sizes and other outcomes rather than evidence to challenge Watt's assertions.

"The accusation that Clemson, its staff and administrators have engaged in unethical conduct to achieve a higher ranking is untrue and unfairly disparages the sincere, unwavering and effective efforts of faculty and staff to improve academic quality over the past 10 years." Sams said in the written statement. "While we have publicly stated our goal of a Top 20 ranking, we have repeatedly stressed that we use the criteria as indicators of quality improvement and view a ranking as the byproduct, not the objective."

What's tricky for Clemson, Sams acknowledged, is that university officials cannot with a straight face say that the rankings do not matter to them; the Top 20 goal has been the centerpiece of President James F. Barker's administration. And the data that Watt presented do not lie: Clemson has, for instance, seen the number and proportion of its undergraduate classes with 10 to 19 students rise to 790 and 32.8 percent, respectively, in 2008, from 356 and 18.8 percent in 2004, while the comparable figures for classes of 20-29 have fallen to 360 (15 percent) in 2008 from 591 (31.2 percent) in 2004.

While Watt attributed that change to a purposeful effort to score better on one of U.S. News's key measures of use of faculty resources -- "Two or three students here and there, what a difference it can make," she said, "manipulation around the edges" -- Sams instead said it had resulted from several faculty-led initiatives in the mid-2000s designed to improve the student experience. One focused on figuring out why students were "washing out" of key courses with D and incomplete grades, and another, Creative Inquiry, connected small groups of students with professors for research-intensive study. Both resulted in smaller classes. "These pedagogical changes ... were just disregarded" in Watt's analysis, Sams said. "It's like none of that ever happened. You can take any statistic and make it look like it's something that it's not, if you don't go and look at other things that were happening at the time."

Clemson's written response directly rebuts Watt's assertion that the university generated several differing versions of faculty salary data for U.S. News and other surveys, a suggestion that particularly drew the ire of some of her institutional research colleagues. "Institutional research has never, not once, produced duplicate faculty salary reports," Sams said. "We report the same data to U.S. News that we report to the American Association of University Professors. U.S. News includes benefits in faculty salary for Clemson and for every national university they rank -- something that Ms. Watt apparently didn’t know."

But the university's statement is largely silent on Watt's most explosive accusation: that Clemson officials, in filling out the reputational survey form for presidents and other top administrators, rate "all programs other than Clemson below average," to make the university look better. "And I'm confident my president is not the only one who does that," Watt said.

The university's statement included a quotation from Dori Helms, the provost and vice president for academic affairs, which said that "neither President Barker nor I would manipulate the data in ranking other institutions. We are proud of other institutions that represent quality higher education in this country and would not want to detract from their reputations."

In the interview, Sams called that accusation "the most troubling" because it "calls into question the personal integrity of people who have high ethical standards.... All I can tell you is that I talked to all three of the individuals who fill out the survey, and they said, 'That does not happen, that is not true.' They were deeply offended."

When a reporter suggested that the easiest way for Clemson to disprove Watt's allegation would be to make public copies of the reputational surveys its officials had submitted to U.S. News, Sams said that the university did not have a complete set of them, and that its officials were concerned that the administrators' responses about other colleges would ultimately be shared with leaders of those institutions. (Inside Higher Ed promised not to release the responses on individual institutions to the public.)

Pressed further, Sams said that "we do have copies of some of them and plan to make them available to you." Clemson had not done so by the time this article was published.

As the firestorm she created with her presentation Tuesday raged around her, Watt herself passed up a reporter's request that she provide evidence back up her assertions. "At this point, I don’t have any additional comments," she wrote in an e-mail reply. "I had hoped to have a discussion about strategic planning, outcomes, and U.S. News with friends and colleagues rather than the 'stir' that has indeed been created."

Watt added that all she was trying to do was to show that "[t]here should be room for multiple points along [the] philosophical continuum" about the role of the rankings and appropriate and inappropriate steps a university should take in response to them.

Sams said she and other university officials believed that in trying to do that, Watt had "done a disservice to people who have really focused on strengthening academic quality and improving research that U.S. News doesn't really care about -- things that would improve the university and as result resulting in a rise in the rankings."

Asked if Watt might face disciplinary action, given that Sams' statement described her comments as "outrageous" and as an "insinuation of unethical behavior that crosses a line," Sams said "I would not think so... There's a lot of unhappiness and a lot of anger, a lot of disappointment. But I think that is where it stops."