Assessing Assessment

SEATTLE -- Margaret Spellings may be secretary emerita, but the assessment and accountability movements-- which of course predated her commission -- are alive and well. And if colleges think they can ignore these pushes, they are seriously misguided. That was the message behind speeches and the announcement of two new national education campaigns here Thursday at the annual meeting of the Association of American Colleges and Universities.

One effort -- further along than the other -- is to create the National Institute for Learning Outcomes Assessment. The institute is attempting to gather information from every college about assessment practices in place in order to produce a national picture of the state of the efforts to measure student learning. Then the institute plans to conduct research on what colleges tell the public about assessment, to study which practices are most successful, and to produce case studies of how assessment works (or doesn't) in certain situations. The second effort -- still formative -- is to create the Alliance for New Leadership for Student Learning and Accountability, which would act as something of a public voice for higher education in national discussions about assessment and accountability.

Both efforts involve to some extent some of the leading national organizations that represent colleges and are receiving backing from such prominent funders as the Lumina Foundation for Education and the Teagle Foundation.

Explaining the efforts, backers said that colleges must get out ahead of these issues -- or others will set up systems that could damage higher education. Higher education "cannot be playing defense," said Molly Corbett Broad, president of the American Council on Education. "That is the message of the day."

The new institute will be led by Stanley O. Ikenberry and George Kuh -- two figures with extensive experience in the politics of higher education and assessment. Ikenberry is former president of the American Council on Education and of the University of Illinois. Kuh is director of the Center for Postsecondary Research at Indiana University at Bloomington, and is best known as the founder of the National Survey of Student Engagement.

Ikenberry raised the question that is no doubt on the minds of those who have hoped that assessment debates might fade with the end of the Spellings era. "Why now?"

The answer, he said, is that assessment has been the subject of debate for 25-plus years "but for a long time has seemed stuck on Page 1." With the visibility brought to the issue by Spellings, with moves by leading college groups to create new accountability systems, and with a proliferation of testing systems, "the next three to five years present a period of significant opportunity" and the future of assessment is "likely to be shaped in important ways," he said. Higher education needs to take the lead, he argued.

Ikenberry also reminded audience members of something they were all talking about anyway: College budgets are being cut everywhere. Reliable, respected systems of assessment and accountability, he said, will help leaders "make wise choices."

Broad said that the current debates about assessment reminded her, somewhat painfully, of discussions 25 years ago about overhead costs paid by federal agencies on research grants. That seemingly arcane topic became controversial when members of Congress questioned some of the expenses universities were reimbursed for, and proposals and counter-proposals flew.

Some in higher education resisted the discussions, saying that what universities did was "so intrinsically valuable that these bean counters in the federal government" didn't understand the issues and weren't worth taking seriously. The result, Broad said, was 25 years of "tremendously onerous" regulations that might have been averted had higher education engaged more successfully in the process.

And she noted that President Obama has said, with regard to numerous issues, that "transparency is the best form of accountability," so the new administration will care about these issues.

Judith S. Eaton, president of the Council for Higher Education Accreditation, also endorsed the work of the new institute. She said she hoped that better knowledge about assessment would improve the relationship between accreditors and institutions, and that a sustained commitment by higher education to accountability would preserve the principles of self-regulation for higher education.

Both accreditors and colleges "need to take a commitment to credibility with the public further than we’ve taken it,” Eaton said. While "some folks from institutions don’t like to hear that [because] they think it’s an implied criticism," Eaton said they need to acknowledge that "the world has changed."

She also said that she hoped the new effort would "strengthen the academic leadership of our colleges and universities." Eaton said that there was considerable strength among those leaders, but that Spellings Commission members and others had encouraged "an undermining" of that leadership by suggesting that colleges are against accountability.

Even if the federal government doesn't increase regulation of accreditation, Eaton said that colleges needed to take it more seriously. "The institutions need to do more to treat accreditation as an ongoing element of quality improvement," not “an abrupt and not always welcome intervention," she said.

Eaton and Broad also endorsed the still embryonic Alliance for New Leadership for Student Learning and Accountability. David Paris, a senior fellow at AAC&U, has been organizing that group and briefed meeting participants on the concept, which was first discussed several months ago at a Teagle-sponsored event in North Carolina. The new group will consist of colleges, associations, accreditors, presidents and faculty members; the idea grew out of discussions at the high point of concerns that the Spellings Commission might do permanent damage to higher education with a "testing regime" or the abolition of accreditation.

While the institute will conduct research and disseminate findings, the alliance will deal with the questions of "What do we say and who speaks for us?" when higher education is asked about accountability, he said. The idea is to promote "professional standards" in policy discussions, "not just out of political self-defense but as a matter of professional responsibility."

The new efforts appear to have a who's who of supporters from among higher education leaders. The advisory board of the new institute includes Broad, Eaton and senior leaders of groups such as the AAC&U, the Association of Governing Boards of Universities and Colleges, the Council of Independent Colleges, the National Association of State Universities and Land-Grant Colleges, the State Higher Education Executive Officers, and the Southern Association of Colleges and Schools.

The audience here seemed intrigued, but not necessarily totally sold on the ideas. AAC&U has been a proponent of assessment and accountability, but has also been criticized many such efforts, calling them oversimplified and potentially harmful. It did seem clear that those asking questions were dubious of at least some of the accountability measures already gaining popularity.

One woman said that she is concerned that the tests currently being pushed for use by colleges seeking to demonstrate student learning outcomes lack "any evidence of validity," even as more institutions endorse their use. “I could buy a bathroom scale and get on it day after day after day and weigh 125 pounds and be very happy, but I’m not 125 pounds," she said. She said that if American society is moving toward the use of standardized tests to measure collegiate learning, "I would implore all of you to give us a standardized test with validity."

Many academic observers may be relieved to know that Ikenberry, in responding, did not vow that the institute would create a new standardized test, and he in fact said that he wasn't looking for single measures. "There are no reliable instruments today," he said. Part of the problem, and part of why the institute is needed, he said, is that no one really knows what people want when they talk about accountability. Said Ikenberry: "Those who call for accountability honestly have no clear idea of precisely what they want -- graduation rates, [Collegiate Learning Assessment] scores, time to degree, or something else."