You have /5 articles left.
Sign up for a free account or log in.
WASHINGTON -- Doctoral departments that have finally figured out just what the National Research Council's controversial evaluation of their programs means, and how to read the numbers, will need to check the data again. On Thursday, the council is issuing revised rankings, based on four "substantive" errors in some of the categories used to evaluate programs.
The council issued a statement on Friday saying that overall rankings won't change significantly for most programs. But the rankings for some programs in some of the categories used for the overall rankings could see real shifts. And that's important because the NRC has said from the start that it is the data sets themselves -- not just the overall rankings -- that should be used to examine programs.
The following are the areas being recalculated and the NRC's explanations of the problems:
- Average citations per publication: "Publications for 2002 used to obtain citations per publication had been mislabeled in all non-humanities fields."
- Awards per faculty member: "The NRC undercounted honors and awards. Data for this variable were re-compiled from faculty lists and the variable was re-calculated."
- Percentage of program graduates with academic career plans: "The response rate to this question, which was calculated from the NSF Survey of Earned Doctorates, varied considerably across programs. It was agreed that a more accurate measure based on survey data was percent of respondents with academic positions or postdocs, not percent of total Ph.D.s. This variable was re-calculated with the changed definition."
- Percentage of first-year students with full financial support: "This variable had been given the value '0' when a program had no first year students. We now use an asterisk to indicate that a program has no first year students. When no data were reported, there is an 'N/D.' "
According to the NRC, the revisions came from a review of various issues raised by departments. In total, the NRC received requests for reconsideration of issues in 450 doctoral programs from 34 institutions. Ten of the institutions had questions raised by 10 or more of their programs.
The NRC rankings have historically been influential with prospective students considering where to enroll, funders seeking where to make grants, and university administrations seeking an independent analysis of where their programs stand compared to others.
This version of the rankings -- shying away from ordinal rankings of departments (giving two ranges based on two methodologies) -- has been controversial from the start. The new rankings have been criticized for being too complicated, and for being so late that many programs have evolved since data were collected. Last month, the American Sociological Association released an analysis blasting the way the NRC's approach compared departments in the discipline. Whether the changes being released Thursday will satisfy anyone remains to be seen.
