How to Judge Community Colleges

SEATTLE -- After leaders of the American Association of Community Colleges revealed details about a new national accountability system to a packed room here Monday, the first question was simple: Who is this system for? Who needs to understand it?

The answer from R. Eileen Baccus, president emeritus of Northwestern Connecticut Community College, and chair of one of the committees developing the system, was also simple: The answers need to make sense for those "who are on our backs."

And that rationale largely carried the day in the discussion at the AACC's annual meeting. The Voluntary Framework of Accountability is a joint effort of AACC, the Association of Community College Trustees and the College Board. As with most new programs in the community college world these days, the money is coming from the Bill & Melinda Gates Foundation and the Lumina Foundation for Education. The idea is to build a community college equivalent of the Voluntary System of Accountability, which is a joint effort of the two main associations of public four-year colleges and universities: the Association of Public and Land-grant Universities and the American Association of State Colleges and Universities. In that system, and in the one being created for community colleges, institutions report a wide variety of data in comparable ways.

While some community college educators have been leery of the effort, given the wide range of community college missions, demographics and funding patterns, the argument of those supporting the new framework is that it should be possible to come up with reasonable ways to judge institutional success, and the briefing here featured details on just how that system might work.

The current thinking of those creating the community college system is to have two main sections: learning outcomes and job training.

Janice Yoshiwara, education services director for the Washington State Board of Community and Technical Colleges, is on the working group focused on learning outcomes, and she described the approach that will be taken.

She said that the standards must be "a set of measures that apply to all colleges," large and small, urban and rural. And she said that the metrics to be used must "make sense to people outside" the community college world -- to business leaders and politicians and parents, among others. She also said it was important to come up with measures that could apply to everyone enrolled at a community college and that they be "reasonable in scope and size."

Specifically, the measures under consideration would include:

• College readiness, focused on how students arrive at a community college and how they become able to reach the college level.
• Success in completing college-level courses.
• Various "credit accumulation milestones," such as earning 15 or 30 credits of college-level work.
• Completion of degree or certificate programs.
• "Overall success indicators" focused on whether individuals achieve whatever their purpose was in enrolling.

There also will be some sort of "learning outcome reporting tool" that would give colleges various ways to report on tests or other assessment tools. At this point, however, Yoshiwara stressed, the framework would not dictate "this test or these scores," but would encourage colleges to make some selections.

She said that for all of the metrics, the working group also wants colleges to provide breakdowns by appropriate subgroups, so that the accountability system would draw attention to whether success levels were the same for minority students or first generation students or various other groups. And she said that more work needs to be done on what information to collect and share about transfer students -- both their success in winning admission to four-year institutions and their performance once enrolled.

For job skills, the accountability system is looking at finding ways to measure all of the degrees or certificates awarded in work-related fields, and also finding ways to measure success in continuing education and in adult basic skills.

Kent Phillippe, associate vice president of the AACC for research and student success, said that the measures are still under review and that colleges' input is essential to the project's goal of having as many community colleges as possible participate. "If we get the measures right, colleges will be participating," he said. And he added that all community colleges benefit from being able to discuss "where we are good and where we are not as good."

After the presentations on the accountability system, most of the comments were strongly supportive. Many of the community college administrators said that they believed this system would provide solid evidence to help shape policy and to answer concerns from lawmakers and others.

But there were some skeptics -- even if they suggested that they were asking questions not out of their own concern, but guessing about faculty reaction. One dean said that as he was listening, "I kept thinking about faculty," and how his faculty members have "a bloodhound's nose" for looking at any national assessment or accountability system, figuring that they will uncover agendas that will lead to "national standardization." He added that including the word "voluntary" in the name wouldn't actually reassure many of them.

"How do you frame this to dispel those kinds of suspicions?" he asked.

Another dean said that when she was in another job, colleagues had talked about "toxic accountability," where various metrics "become ends in and of themselves" and encourage "teaching to the test." It's very easy for faculty members, she said, to view accountability as "another way to take time away from students."

Baccus reiterated that it is important for the colleges themselves to be first to define accountability for community colleges. With growing federal and private sector interest in community colleges, she said, there will be accountability systems created. "Do we want someone else to impose these measures, or do we want to say that here are measures we can endorse?" she said.

Phillippe agreed, and said that -- at the same time -- it was key to distinguish between federal and national accountability. The idea is to have national comparability, but not to have a federal requirement, he said. In fact, he said that getting a system like this up and running might convince some of those who want more federal oversight that it isn't needed.

Still, he acknowledged that once a system is in place, it could be used in ways that were never intended. "We wouldn't use this system to rank colleges, but that's something that probably will happen."

Any who doubt Phillippe might consider the Community College Survey of Student Engagement, a highly respected assessment tool. In 2007, when The Washington Monthly wanted to rank community colleges, the magazine ignored the survey's repeated explanations that the project wasn't designed for rankings, and used it for them anyway.