More Rankings Rigging

How many members of the National Academy of Engineering are on the faculty at the University of Southern California?

This might seem like a straightforward question, but it's anything but when you add in the politics of rankings. USC's Viterbi School of Engineering maintains a list of 34 faculty members it says are in the academy. And when reporting to U.S. News & World Report, which uses NAE members on the faculty as one criterion in its rankings of top engineering graduate schools (where USC landed at No. 7), Southern California claimed 30 members.

But according to the National Academy of Engineering, USC has only 22 members on its faculty.

USC provided three different explanations to Inside Higher Ed when asked why it was claiming more faculty members as NAE members than it appears to have. First, an official said that some faculty members work elsewhere, in the business world for example, and teach part time at USC. Told that such faculty members wouldn't meet the U.S. News criteria (which count as faculty members those who are full time and on the tenure track), the officials responsible for handling the rankings referred questions to the dean, who said that all 30 National Academy of Engineering members claimed by USC as faculty members teach full time.

Then, when Inside Higher Ed provided USC with a list of 17 of the 34 supposed faculty members who did not appear to be full-time faculty members after all, USC said it was reviewing its procedures for counting, and had informed U.S. News of a possible problem. Further, U.S. News confirmed to Inside Higher Ed that it has made no attempts to verify the reports universities file on such matters as National Academy membership, and simply assumes universities are providing accurate information.

The problems with USC's submission come days after a Clemson University official disclosed the extent to which she said the university attempts to manipulate the system to improve its rankings, renewing the debate over whether the rankings system encourages and rewards gaming of the system as opposed to sound educational decisions.

Inside Higher Ed was first alerted to the possible problems in the USC figures by postings on the Web site College Confidential, where Sam Lee, a graduate student, suggested that something was wrong with the National Academy membership numbers. Lee noted that USC came out in the No. 7 spot even though its rankings in all of the engineering subfields were lower than that, and he noted that the area that favored USC was National Academy membership -- and suggested that problems with that calculation might be unfairly raising USC's score. So Inside Higher Ed contacted U.S. News to find out how many academy members USC had claimed (30), the definition U.S. News uses (full time, tenure track faculty member), and the person at USC who prepares the report.

That person is Margery Berti, an associate dean at the engineering school. In a first interview Friday, she said that the figure of 30 was accurate and that the reason that was greater than the figure listed by the academy was that many engineering professors have jobs elsewhere -- as consultants or at their own businesses. The National Academy asks members for their "primary" employer and identifies them this way, so USC just wasn't the primary employer of some of these people, she said.

Told that U.S. News counts only full-time tenure-track faculty members (presumably people for whom USC would be a primary employer), Berti said that the only person who could answer questions on the matter was the dean, Yannis C. Yortsos. Berti then got Yortsos on the phone, but refused to let this reporter ask questions until she explained the U.S. News policy of which she had just been informed. At this point, Yortsos insisted that all of the NAE members USC claimed as faculty members were full-time faculty at the university. Asked why the National Academy had a lower number, Yortsos said that "there are cases were people who are elected as a different institution [before becoming a full-time faculty member at USC] and they do not change their affiliation."

He was asked several times and repeated several times that all 30 faculty members claimed by USC as NAE members were full-time faculty members.

That led to an examination of the list of 34 claimed by USC on its Web site as faculty members in the Academy. An examination of this list found that:

• Three have full-time jobs at USC that rule out the possibility that they are full-time faculty members. (These three are the university's president, provost and engineering dean.)
• Four have emeritus status.
• Ten are either listed elsewhere as having full-time jobs (in businesses generally) and/or are not listed in USC directories as having offices at the university.

To give two specific examples, USC lists Wanda M. Austin as a faculty member who is an academy member. Austin earned her doctorate at Southern California, but she is the CEO of the Aerospace Corporation. Or there's Paul Kern, whom USC includes on its list and who does indicate in his biography that he is an adjunct there. Kern spends most of his time, however, working with the international consulting firm the Cohen Group and holding an endowed chair at West Point.

In some cases, those listed as USC faculty members with academy membership appear to have been added to the roster a bit after their peak research years. Simon Ramo -- whose career has been distinguished in developing weapons systems (the R in the company TRW is for his name) -- wasn't named to the USC faculty until last year. He was 94 at the time. The USC directory does not indicate any contact information for him on campus.

This list was then sent to USC, leading to a new response from Berti. She noted the many accomplishments of the engineering program and then went on to say that "everyone listed on the school's NAE Web page has a faculty appointment in the school. As you suggest, not all of them are full-time, tenure-track, hence we did not report to the USNWR all those listed." But in what appears to be an acknowledgment that the number qualifying for the U.S. News calculation may be considerably lower than what the university submitted for the rankings, Berti wrote that "thanks to your question the dean requested that we scrutinize our USNWR submission. We are in the process of reviewing the NAE faculty count as defined by the USNWR instructions. We already contacted USNWR to report this and will submit to them any changes as soon as we can verify them."

On Sunday, Leslie DaCruz, executive director of communications for USC's engineering school, sent an e-mail in which he said that those faculty members who don't teach full time still add to the quality of the program. "They participate in courses, mentor students, mentor faculty, advise on research directions, and more. They have close and highly supportive relationships with the Viterbi School. Of course some are not full time, and some do not have on-campus addresses or phone numbers. These are all extraordinary figures in the profession, who add a great deal to our academic community."

Robert Morse, who directs the rankings for U.S. News, said that if USC indeed has significantly fewer faculty members in the academy than the university claimed, that could well lower the engineering college's ranking. But Morse said he couldn't tell how much without knowing the actual numbers. Morse said that U.S. News defines the way it counts faculty members (as full time, tenure track), but doesn't seek to verify the numbers submitted by universities.