New Measure of Success

Two issues on which many higher education experts agree are that completion rates matter and that the federal methodology for measuring graduation rates is seriously flawed. To date, however, no alternative system for measuring graduation rates has gained widespread currency in discussing the performance of colleges.

Today six higher education associations (representing two-year and four-year institutions, public and private) are endorsing a new system called the Student Achievement Measure, with the hope of changing the debate about measuring college completion. While individual colleges will have to decide whether to participate, the endorsement of associations representing every nonprofit sector suggests at least the possibility of broad participation. The major financial backer for the new effort is the Bill & Melinda Gates Foundation, which has in recent years played a prominent role in discussions of college completion.

The Student Achievement Measure (or SAM) involves looking at a number of student cohorts not measured by the federal rate and reporting on multiple success measures for each of them, using different time frames than the federal system.

The federal methodology examines whether first-time, full-time students have graduated with a bachelor's degree after six years or an associate degree after three years. Many say that the system works well for only some institutions -- generally residential colleges that educate almost entirely traditional-age college students who enroll full time and stay at one institution. But for many colleges and universities, that population is practically nonexistent, and for many others, that population is shrinking.

So institutions that educate large number of students who move in and out of several institutions, or who transfer without a degree, can easily have a majority of their students not counted by the federal system. This is commonly the case at community colleges. Others have criticized the six-year time frame of the federal rate, noting that many students and parents consider success to be a bachelor's degree in four years.

SAM would consist of two systems, each of which would study two cohorts.

For bachelor's degrees, SAM would examine data on:

• Full‐time students attending college for the first time.
• Full‐time students who transferred in to the reporting institution.

For each of those groups SAM would record the students who:

• Graduated from the reporting institution.
• Are still enrolled at the reporting institution.
• Transferred to/graduated from one or more subsequent institution.
• Transferred to/are still enrolled at a subsequent institution.
• Have unknown current enrollment or graduation status.

And the data would be reported for periods of four, five and six years after enrollment.

For associate and certificate programs, SAM would examine data on:

• Full‐time students attending the reporting institution for the first time (including both new students and students who transfer in from another institution).
• Part‐time students attending the reporting institution for the first time (including both new students and students who transfer in from another institution).

For each of these groups, SAM would record the students who had:

• Graduated from the reporting institution.
• Are still enrolled at the reporting institution.
• Transferred to one or more subsequent institutions.
• Have unknown transfer, current enrollment or graduation status.

The associate and certificate data would be provided only for periods of six years after enrollment.

Colleges are being invited to start submitting data this fall, and initial response is positive, said Christine M. Keller, associate vice president for academic affairs of the Association of Public and Land-grant Universities, which has championed the new system. "Everyone's been frustrated by the federal rate for a long time," she said.

Keller said it was "really important" that the different sectors of higher education were moving on the issue together. "All of us had been working on our own," she said, and the common push means that policy makers, students and families can turn to a single location -- the new SAM website -- for full data on completion.

Down the road, she said, it is possible that additional cohorts of students -- such as part-time students in bachelor's programs -- could be added to SAM. She said it was a "pragmatic decision" to start SAM without every possible cohort, and that she believed the organizers would soon look at other groups of students that could be counted.

Community colleges have been particularly vocal in recent years in criticizing the federal rate. "It is no secret that the federal graduation rate obscures as much as it reveals. This effort will show that higher education is performing better than many people think -- even if graduating more students, particularly at community colleges, remains an imperative," David Baime, senior vice president for government relations at the American Association of Community Colleges, said via e-mail.

Baime added that "national policy is hopefully headed in the direction of these and related measures."

The use of six years as the time frame for community colleges is designed to be consistent with the AACC's Voluntary Framework for Accountability, in which community colleges report on various measures of performance. Baime said it was important that SAM be consistent so that colleges could be recruited to both efforts without having to recalculate data. He said that six years would count many graduates ignored by the federal methodology.

The college groups endorsing SAM are the American Association of Community Colleges, the American Association of State Colleges and Universities, the American Council on Education, the Association of American Universities, the Association of Public and Land‐Grant Universities and the National Association of Independent Colleges and Universities.