Validation Required

TORONTO -- As several colleges over the last 18 months have admitted to submitting false data to U.S. News & World Report and other organizations that compile rankings, U.S. News has insisted that there is no broad problem or need for a new system to verify the accuracy of submissions.

But on Saturday here at the annual meeting of the National Association for College Admission Counseling, the ethics committee of the association announced an additional mandatory requirement to respond to all the false reporting. Also on Saturday, The Washington Post published an article revealing that Washington & Lee University has been reporting its admissions rate based on number of applications started, not the applications completed, a choice that lowered the institution's admit rate, making it appear more competitive than it is.

The association already had requirements that member colleges present information about their institutions accurately. That requirement has now been clarified to say that colleges must have "an official policy regarding the collection, calculation and reporting of institutional statistics. This must include a process for validating all institutional data."

"Validating" was not defined, and NACAC officials said that colleges could fulfill the requirement with an outside audit -- as some have suggested as the best solution to assure accuracy. But the requirement could also be met by having one other person review the work of the person who prepares statistics for submission.

In several of the cases involving fabricated data, a single person was able to compile and submit the false information.

How to Measure Admissions Rates

The Post article reported that about one-sixth of the applicants Washington & Lee reported never finished their applications. When only completed applications are counted, the institution's admit rate rises from 19 percent to 24 percent (still leaving it among the highly competitive colleges in the United States).

Bill Hartog, the dean of admissions at Washington & Lee, denied that the university was trying to deceive anyone or inflate rankings. "We don't even think about that stuff," he told the Post.

The Post quoted several other elite liberal arts colleges as counting the way Washington & Lee does while many others count only completed applications or only a small subset of incomplete applications. While Washington & Lee and other institutions that counted incomplete applications in material they submit for rankings said the practice was legitimate, there are signs it may not be -- at least when the data are being used to compare institutions.

The Common Data Set (which is used by U.S. News in its rankings) defines an applicant as one who has "completed" the application, and the Education Department says that one becomes an applicant when he or she has "fulfilled" all of the requirements of the application process.

Further, in four of the well publicized recent cases where colleges have admitted to submitting incorrect data to U.S. News and others, their investigations have said that one of the things they were doing that inflated rankings was counting incomplete applications. Those cases include three undergraduate colleges are Claremont McKenna College (which also fabricated in other ways), Dominican University of California, the University of Mary Hardin-Baylor and the business school at Tulane University.

The admit rate counts for only a very small share of the U.S. News methodology in evaluating undergraduate institutions (1.25 percent), but the admit rate is among the highlights of many rankings (reported separately) and is commonly cited in comparing colleges.

Robert Morse, who heads the rankings operation at U.S. News, said that the Common Data Set counts an application as complete as "actionable," meaning that the college can accept, deny or waitlist the applicant. So if a college opted to reject all of those with incomplete applications, it would meet the definition and could count all those applicants. Morse said he would be looking into the issues raised by the Post article.

Via e-mail to Inside HIgher Ed, a spokesman for Washington and Lee said that while the university stands behind its practices, "we always review our processes and practices and will continue to examine our policies and practices in the future, especially in light of the impact technology is having. We will not shy away from addressing straightforwardly the questions now being raised and adjusting if necessary. In fact, our decision to shift entirely to the Common Application for the 2013 admissions cycle was based, in part, because we were finding that more students than we would want were not sending all their material on time. As a consequence of that change, the number of students who were missing an element was cut in half from the previous cycle."