Dubious Data

Last week, The Washington Post published an article compiling U.S. Department of Education data on the number of rapes reported on college campuses. “These colleges have the most reports of rape,” read the headline of the article, which included a sortable chart of the data, with Brown University and the University of Connecticut topping the list.

While The Washington Post noted that the high number of reports could be a positive development, indicating that students are feeling more comfortable coming forward about their assaults, other publications used the chart to create stories that ranked institutions by how many rapes they reported. Dartmouth College’s 42 rape reports in 2014, for example, inspired the headline “Dartmouth comes second in national study of reports of campus rapes.”

While it makes for good headlines, researchers and advocates say using federal reporting data to assess the prevalence of campus sexual assault or to rank the relative safeness of individual colleges is ill advised and even irresponsible.

“It is really misguided to use sexual assault reports as rankings, because schools with higher rates are actually doing a better job of encouraging reporting and addressing the issue,” Laura Dunn, founder and executive director of the victims' advocacy organization SurvJustice, said. “By ranking schools with higher rates as unsafe, the media's uninformed coverage is actually discouraging schools from better addressing campus sexual assault. We don't want to push reports into the shadows; we want [assaults] to be reported and dealt with appropriately.”

In a statement responding to questions about the criticism, the Post defended its project.

"We stand by our report and took great care in our analysis to provide context to readers about the federal data on reports of rape at each school," Josh White, the Post's education editor, said. "We noted prominently in the story that victim advocates say it is a positive trend that growing numbers of students who may have experienced a sexual assault are stepping forward to tell authorities about incidents that in years past might have gone unreported. We quoted university officials making similar points about the reporting totals on their own campuses, and we noted that there can be questions raised at schools that have a low number of rape reports."

The Data

Under the federal Clery Act, colleges and universities are required to collect and disclose statistics about crimes that occur on campus. That includes domestic violence, stalking and rape. In previous years, rape was included under a broader category of "forcible sex offenses." This year, because of changes to the Clery Act in 2014, rape reports are listed separately. The numbers are made publicly available by the U.S. Department of Education and on the colleges’ websites.

As Clery data only include crimes that happen on campus, many institutions in urban areas where most people live off campus regularly report zero rapes. At the same time, this can mean private liberal arts colleges in small towns with large on-campus populations can report disproportionately high rates of sexual assault. Reed College and Wesleyan University, for example, had the highest total reports of rape per 1,000 students, according to The Washington Post's analysis.

Clery data should not be used as a tool for comparing or ranking institutions, said Mary P. Koss, a professor of public health at the University of Arizona and a pioneering researcher on the prevalence of campus sexual assault.

“It is a completely, totally invalid assumption,” Koss said. “In some respects, high numbers can be good. If you’re revamping your approach to sexual assault, you would actually expect the number of reports to go up. But even those high rates are not credible, as they are just the number of reports, not actual assaults. The bigger story is looking at those numbers in the context of how many rapes are being identified by climate surveys.”

Colleges are increasingly using such surveys to help determine the prevalence of campus sexual assault. The surveys ask students to share their experiences with -- and attitudes about -- harassment, stalking and gender violence.

The juxtaposition between climate surveys and Clery data isn’t perfect, as climate surveys point to a number of students, including those who never reported what happened to them, while the federal data points to a number of reported incidents on campus. But comparing the two can help reveal just how few assaults are actually reported on many campuses.

“Clery data tells us one thing: how many rapes are reported by students and honestly recorded on a Clery report,” said John Foubert, a professor of higher education and student affairs at Oklahoma State University and founder of the sexual assault prevention program One in Four. “The only use I see for it is to gauge how far an institution needs to go in closing the gap between reported rape and the actual rate, which needs to be determined by anonymous surveys.”

Brown and the University of Connecticut, in The Washington Post’s analysis, tied with the highest number of reported rapes. “Brown, UConn rank first in this troubling campus statistic,” a Boston Globe headline announced this week. With 43 reported rapes, Brown does indeed rank first, but a survey created by the Association of American Universities and conducted at 27 colleges last year suggests that the university does not have higher rates of rape than other institutions.

It also suggests that Brown may do a better job at encouraging victims to come forward than colleges with fewer rape reports.

At Brown last year, about 2 percent of female undergraduate students and less than 1 percent of male undergraduate students reported experiencing completed or unwanted "nonconsensual penetration." That’s about 90 undergraduate students, compared to the 43 reports collected by the Education Department. In other words, half of rapes experienced by Brown students that year may not have been counted in the federal data.

For institutions that have few reported rapes, that gap can be even wider. According to the Clery data, the University of Oregon, for example, only reported six rapes in 2014. The AAU survey found that more than 500 Oregon undergraduate students said they had experienced attempted or completed nonconsensual penetration in the last year.

The data also may suggest that some colleges are doing a good job at protecting students -- even if evidence casts doubt on those assumptions. Hundreds of colleges and universities reported no rapes at all. 

Among colleges with the fewest reports was Baylor University. The university reported just four rapes in 2014, but last month Baylor's Board of Regents fired its head football coach and forced its president to resign over allegations that the football program had made sure sexual assaults involving players were not reported to the correct officials.

While comparing campus climate surveys with Clery data can indicate which colleges need to improve how they encourage students to report rape and sexual assault, Anna Voremberg, managing director of End Rape on Campus, said she would still caution against ranking institutions.

“I think there's a misunderstanding of the data and a misunderstanding of the issue,” Voremberg said. “I don’t think we should be comparing Dartmouth to Baylor or Brown to Oregon. Those are really different schools with very unique problems. We need to understand what’s going on at each campus.”