Institutional Research Roundup

CHICAGO -- Institutional researchers are higher education's version of a utility infielder. That doesn't mean they lack expertise: They specialize in bringing data to bear on issues and problems, and explaining and interpreting those data to campus constituents who often come at the information from widely varying viewpoints. Their versatility comes, though, in the wide range of subjects they touch and of decisions over which they have some influence.

Given that eclectic role, the annual forum of the Association for Institutional Research typically covers a plethora of topics, and this year's meeting, the organization's 50th, is no exception. But it is also true that examining the forum's agenda usually offers a sense of which issues are keeping institutional leaders up at night, since those are often the topics that presidents and provosts and other campus officials have asked their data gurus to dive into.

Not surprisingly, given the emphasis that policy makers are placing on college completion and the fiscal realities that make every lost student a liability, retention and student success were all over the AIR agenda. Roughly a third of the 375 sessions related to institutional efforts to measure or improve students’ academic progress in higher education.

In one such session, Roger Mourad, director of institutional research at Michigan’s Washtenaw Community College, compared the characteristics of students who transferred from his institution and then graduated from a four-year college to those who transferred and did not earn a bachelor’s degree.

The study would help to shed light, Mourad said, on what he said remains a “very viable debate nowadays”: “Whether community colleges are democratic institutions operating as gateways to four-year institutions, or do they end up diverting students away from four-year bachelor’s institutions?”

Mourad’s study, which examined students who entered Washtenaw for the first time in 2000 and followed for eight years those who transferred to a four-year institution, found that about 44 percent of all transferring students graduated (with significantly higher proportions of transfers graduating from the University of Michigan than from Eastern Michigan University and other institutions).

Students were more likely to complete their bachelor’s degrees if they earned more credits and had higher grade point averages at the two-year college before transferring, as one might expect, Mourad said. But every additional semester they spent at Washtenaw actually reduced their odds of earning a bachelor’s degree, he said. “Students who were more immersed academically at the community college over a shorter period of time were better prepared to succeed at four-year institutions,” he said.

Why might staying longer at the community college actually reduce their likelihood of completion at the four-year institution? Mourad and the audience offered several theories, including that students “become too comfortable with the small class size, the easier access to faculty members,” and other nurturing elements of the two-year environment, or that they get used to the “less competitive” environment (marked by “easier grading”) that they may find at two-year institutions. “When they hit the four-year institutions, do they have transfer shock?” he wondered.

Diane Dean, an assistant professor of higher education policy at Illinois State University, came at the question of bachelor’s degree completion from another angle.

Amid growing interest among state policy makers in trying to limit fast-rising tuition rates, she examined whether state guaranteed tuition programs affected retention and completion rates.

Looking at comparable students and institutions in Illinois (which has a guaranteed tuition program) and those in surrounding Great Lakes states, which do not, Dean found that Illinois’s program had had insignificant effects on the success of its students at public universities. That may be, she speculated, because guaranteeing students a tuition rate may improve predictability of what students pay, but it doesn’t, by itself, make college more affordable for those students.

A Search for a Better Way

Many if not most sessions at the institutional researchers’ meeting involved campus IR officials presenting the results of studies they’ve conducted, with the goal of shedding light on local issues or problems.

One session Monday had a very different purpose: providing a forum for a group of college officials grappling with a common problem: the failure of the federal graduation rate to capture what’s happening on campuses filled with adult students.

Chris Davis, vice provost of institutional effectiveness at Chicago’s National-Louis University, said that many campuses like his were trying to find their own alternatives to the federal rate, which by focusing exclusively on full-time, first-time students captures a tiny fraction of the students at many adult-serving institutions. National-Louis has begun contemplating a series of indicators to measure its own students' success, such as looking separately at the graduation rates of students who transfer into the university with 15 or more credits and those who enter the university with 45 or more credits.

But “rather than do this in a vacuum, I thought it would be useful to get a discussion going among people who are also wrestling with this problem,” he said.

IR officers from a mix of institutions -- public and private, nonprofit and for-profit, primarily online and mostly bricks and mortar -- welcomed the discussion, which Jim Miller, of New Hampshire’s Granite State College, said was “kind of like a support group.” His institution, which had just 13 full-time, first-time students in one recent year, has begun focusing on year to year retention, which “gives us a much more realistic picture” of how students are faring than does the federal graduation rate.

The ultimate goal of such a group, which Davis said he would try to keep alive through the use of a Google group, should be to give the growing number of institutions that serve significant numbers of “nontraditional students” (a phrase that means less with each passing day, they acknowledged) “at least some basis for comparing ourselves to one another, even as we keep some individual measures that may work best for our own places,” said Jan Lyddon, director of institutional effectiveness at Franklin University, in Ohio.

Breaking News: NRC Rankings Coming 'Soon'

Charlotte D. Kuh gave a very thorough presentation Monday about the National Research Council's long-delayed ratings of doctoral programs, so much so that she left no time for the audience to ask any questions. Given how unsatisfying Kuh’s message probably was to many in the audience, a cynic might wonder if she planned it that way.

Beyond what Inside Higher Ed reported last month, Kuh offered relatively little in the way of new information about the status and shape of the council’s doctoral rankings, which have been put off enough times to raise questions about their relevance. Heads were seen shaking when she told the assembled that the rankings would be released "soon."

Kuh did reveal a few nuggets. She said the NRC will issue a "report" on the state of doctoral education, but that the ratings of individual programs would not be included in the report itself; instead, they will be made available in Excel spreadsheets on the Web. The decision not to include the institutional ratings in the report is purposeful because of the council's desire not to oversimplify the findings. "We don't really like the 'Who's No. 1 question;' " she said.

She said that the rankings will not be replicable by other researchers because they will be based on 500 different estimations, and those estimations will not be made available to the public.

Kuh also provided some examples of how differently programs might fare in the different types of ranking that the council now plans -- some top programs might fare well over all, but show relatively poorly in the quality of the student experience they provide, for instance.

Though Kuh did not take questions, she made clear -- even as she insisted that the rankings would still be useful and relevant, noting that there's "a tradeoff between speed and accuracy" -- how eager she is to move on. "I can't tell you how much I'm looking forward," she said, to the report's release.

Soon.