The Assessment Gap

PHILADELPHIA -- Audience members here at one session of the annual meeting of the Middle States Commission on Higher Education were asked to talk among themselves and then to share their biggest challenges with assessment. Among them: "using assessment results," "communicating assessment results," "resistance of faculty," "herding the cats," and "too much of an emphasis on process and not enough on purposes."

These and other answers were variations on a theme: Everyone is doing assessment, but at many colleges, not many educators are using the results. For regional accreditors like Middle States, it's hard to take much satisfaction in the institutional consensus about having assessment activities in place if they don't prompt change, given that the argument accreditors and others have made about assessment is that it should spur improvements.

If assessment is only done to satisfy accreditors, after all, it may not be done or taken seriously when there's not a self-study to be produced.

"We don't want faculty to do this only because they know that they are coming," said Stephanie Schull, coordinator for undergraduate assessment at Temple University, with the clear implication that the second "they" was made of up of those in the audience -- the deans, provosts and faculty members who serve on the committees that conduct accrediting reviews. And to judge from the presentation by Schull and a colleague here -- on how to manage assessment "top-down" and "bottom-up" -- many of those leading assessment efforts want their faculty members and administrators not only to do it, but to see the value in it.

The discussion here reflects a growing frustration that assessment efforts are hindered by a lack of follow-through, once data are collected or forms are filed. In October, when assessment experts gathered to mark the 10th anniversary of the National Survey of Student Engagement, which is being used at a growing number of institutions, similar complaints were voiced.

At the Middle States meeting, the Temple officials offered strategies for winning faculty involvement and follow through:

• Recognize differences among departments. Jodi Laufgraben, associate vice provost, noted that many programs at her university have specialized accreditation requirements, and said that, generally, faculty members in those programs are more involved than others in accreditation and assessment requirements. "We sort of have an accreditation culture and a non-accreditation culture," she said. That means a decentralized approach may be more successful, given differing degrees of acceptance or resistance to assessment. To that end, Temple insists that every college or division do assessment, but doesn't dictate a single approach, she said.
• Publicize success stories. Temple requires all departments to have periodic program reviews, and some professors entered that process "kicking and screaming," Laufgraben said. But attitudes changed when some departments were able, based on those reviews, to grow. "Once you have a department saying 'We have hired two new faculty as a result' or 'We changed our graduate program as a result,' it made a big difference," she said. Faculty members will respond to seeing assessment make a difference, she suggested.
• Start with the basics. Schull said that assessment experts like those in the room need to have realistic first steps. "Don't try to get from A to Z. Just have a plan to get to Z," she said. Many professors view assessment as "very esoteric and very bureaucratic," she said. They can be approached by encouraging them to "tell stories of what they do to improve," and they will find in most cases that they are doing assessment (but may not be recording or analyzing it in any way).
• Reward, don't punish, the identification of flaws. Colleges should link budgeting to evidence that departments are engaged in assessment, Schull said. But they need to "be careful" to do it in a way that doesn't encourage departments to hide problems. Using assessment to identify and fix a problem is something to be rewarded, she said. "If people think that if they uncover a problem they are going to get dinged, you won't get any assessment or you'll get cooked assessment," she said.

The session at Middle States didn't feature anyone from Temple's faculty. Reached later, Arthur Hochner, president of the faculty union (an American Federation of Teachers affiliate), said it was fair to say that Temple's efforts are decentralized. "There is no general assessment regime," he said.

But he said that despite being decentralized, and there being many seminars and programs, he wasn't sure that many faculty members actually felt close to the efforts. "I don't think it's as effective as anybody would like," he said. "It seems to me that a lot of it is driven by administrators who come up with ideas, and try to get faculty to go along with it, and faculty don't have ownership of the process."