Consensus (or Not) About Comparability

As she formally opened the Education Department's forum on higher education accreditation Wednesday, Vickie Schray, an aide to Secretary Margaret Spellings, clearly sought to reassure those participants and onlookers who have feared that the department has an aggressive agenda up its sleeve to revamp higher education's system of quality assurance, in line with the recommendations of the Secretary of Education's Commission on the Future of Higher Education. "Today's meeting is not about criticizing," she said, adding, "We're not here to reach consensus about any of this."

Yet two hours later, after leaders of the various small group discussions reported back on what had been said around the eight-person tables, Schray seemed almost giddy that many of the participants had acknowledged the need for colleges to use and report common measures of what, and how successfully, their students learn. "Nobody shied away from comparability," Schray said. And Spellings herself, when she addressed the group, said she too was "struck by the commonality of thought here."

Some participants agreed. Margaret A. (Peg) Miller, a higher education researcher at the University of Virginia who spoke forcefully near the start of the meeting about the need for college officials to get with the Spellings commission's program, said afterwards that she had been pleasantly surprised by "how far higher education seems to have come in seeing the value of publicly reported data" that can be compared among "peer institutions."

Miller qualified that statement, though, by noting that the nearly 60 college officials, accreditors, business leaders and other invited to participate in Wednesday's forum came largely from the public sector and excluded all but two representatives of private nonprofit colleges, the sector of higher education that has expressed the most qualms about the possibly of government-mandated requirements that colleges use similar measures. "The outliers are not here," Miller said.

Even so, the call for comparability was far from unanimous. "At our table ... there was a great deal of concern about comparability," said Terry W. Hartle, senior vice president for government and public affairs at the American Council on Education. "I think institutions can and should be pushed to do more on this, and to be more transparent about it. But any effort to force or propose some sort of comparability standard will be enormously complex and exceedingly controversial."

Added Judith S. Eaton, president of the Council for Higher Education Accreditation, a membership group that coordinates accreditation nationally and recognizes more than 60 regional, national and specialized accrediting groups. "They may claim that they have agreement about moving to common templates. But we fought over that tooth and nail in our group."

Wednesday's discussion was the most tangible step so far by the Education Department to begin carrying out the recommendations of the commission that Spellings appointed to examine the current state and the future of American higher education. Although the panel offered ideas on a wide range of topics and issues, including increasing need-based financial aid for students, expanding access to underrepresented students and significantly greater accountability, department officials have made accreditation their first order of business for two reasons: (1) they believe accreditation can be an important lever in achieving other changes, and (2, and more pragmatically) they believe it is an area where they can achieve the most without having to seek Congress's approval.

Department officials have already floated the possibility of using an upcoming process of federal rule making to carry out changes in the regulations governing accreditation. They have also asserted frequently that the standards used by the department's National Advisory Committee on Institutional Quality and Integrity, which provides to or withholds from accreditors the authority to approve individual institutions or academic programs, could be used to force changes in how accrediting groups operate. That panel next meets next week, and some accrediting officials said they expected the committee to begin to flex its muscle then, perhaps holding some accrediting groups' feet to the fire on learning outcomes.

At Wednesday's meeting, department officials struck a collaborative rather than confrontational tone. "I'm here to allay some of your fears," Spellings said to the group. "I fully understand that this is not a place where one size fits all is either desirable or workable," she said about the measurement of learning outcomes. "I value the great diversity" of higher education.

But she also made clear that while department officials were hopeful that academics themselves would get behind the movement to use the accreditation process to better measure how well students are learning -- "we're going to do this with you, not to you" -- time was of the essence. "There are some things that we can do, and let's not let the perfect be the enemy of the good," she said. "This is something we need to get about quickly."

Many participants in the meeting did indeed seem open to the need for significant changes in accreditation, particularly in how hard the process pushes institutions to show that they are succeeding in educating students. Many accrediting and college officials have objected to the constant characterizations by the Spellings commission and department officials that they have done little to assess how well students learn, noting that accrediting groups have been much more aggressive in recent years about insisting that colleges assess their own performance on that score.

But it is one thing for a college to use its own or external measures to assess how well it is doing, and quite another for individual institutions -- and higher education generally -- to show the world that students are learning, which is the stated goal of the Spellings panel and department officials. The only way that colleges can be held accountable for their performance, this line of thinking goes, is if they use measurements or tools that can be compared to those used by other institutions, too -- and if the results are then made public in a way that allows consumers and policy makers to judge how they're doing.

"In the past years, there has been tremendous development in self-improvement, in the use of student learning outcomes to energize campus discussions," said Peter Ewell, vice president at the National Center for Higher Education Management Systems and one of the country's leading experts -- for nearly 25 years, he noted with some irony -- on higher education accountability. But "asking campuses to do assessment is not the same thing as assuring" that all graduates are achieving and of making it public. "Addressing the issue of comparability requires external benchmarks," and while standardization -- in the form of testing -- "is the easy answer," it may not be the only one, Ewell said. He cited externally validated electronic portfolios and "discipline-based capstone" experiences as other possibilities. 

Participants through out a slew of ideas for how higher education -- perhaps through the accrediting process, perhaps not -- might develop a set of student learning measures that might be comparable from one institution to another (most likely within sectors or groups of institutions with common missions: public research institutions, say, or for-profit institutions that serve adults). Several cited the voluntary system that the National Association of State Universities and Land-Grant Colleges and the American Association of State Colleges and Universities are jointly developing, while Jane Wellman, a higher education consultant, said American college officials might consider defining a common understanding across institutions of expectations for degree level work, like the National Qualifications Framework that European countries are crafting.

Others suggested that accrediting groups might develop menus of acceptable measurement tools, to which institutions could seek to add their own measures upon approval by the accrediting groups or perhaps a new third-party entity. Still others suggested that the department should focus instead on building up its own data collection system, constructing a "huge database" of information on student outcomes and other issues.

For all the brainstorming, though, it was difficult to walk away from Wednesday's meeting with a clear sense of where this conversation was heading, and whether there was anywhere near a consensus on even which direction the next baby steps should go in.

In an interview with reporters, Spellings, for her part, acknowledged that that was to be expected, given that this was an "opening discussion." She said that she would "take [college and accrediting officials] on good faith that they're willing to get to work right away, as I am," and that she had "no predetermined timelines" for when she needed to see progress. But she implied, clearly, that she would not wait forever.