A Conversation Starter

INDIANAPOLIS -- Nearly four years ago, the Lumina Foundation and some very prominent names in higher education policy circles unveiled a project with a nice, humble goal: to better and more clearly define what college degree holders should know and be able to do.

Inside Higher Ed described the Degree Qualifications Profile this way in January 2011:

The profile, around which Lumina officials plan to begin a several-year discussion in which colleges, accreditors and other groups will test and refine it, is intended to establish, in more specific ways than has historically been the case, what the recipients of associate, bachelor's and master's degrees (regardless of discipline) should know and be able to do.

The degree framework, [Lumina President Jamie Merisotis] said, is designed to help develop ... a shared understanding across majors, programs and institutions.

Rather predictably, the Degree Qualifications Profile provoked both significant enthusiasm and concern aplenty -- for many of the same reasons.

Lumina officials went out of their way at nearly every turn to say that they were not trying to create a national accountability framework, but the foundation’s close ties to policy makers (many of whom favor such systems) fed mistrust. Foundation officials also described the work as potentially "transformational" -- rhetoric that suggested the sort of dramatic change that tends to be threatening to many academics when they haven't been included in or aware of the conversation.

Advocates for the profile emphasized that it would take hold only if faculty members “owned” the proficiencies, and Lumina vowed to seed discussions on campuses at the faculty level. But the foundation's track record of excluding professors from much of its work was well-established by that point.

Its efforts to frame the DQP as a grassroots effort were further hampered by its big-dollar grants to accrediting bodies, associations and others to experiment with the profile, spurring doubts about the genuineness of the experimentation.

Today, at an event here in its home city, Lumina is unveiling a refreshed version of the Degree Qualifications Profile, which foundation officials say reflects feedback gleaned from work done with the document by faculty members on about 400 campuses. Major changes are few -- a greater focus on quantitative reasoning, new proficiencies related to ethical reasoning and global learning -- and foundation officials describe it as "more an enhancement than a revision. "The fundamental strength of the DQP -- succinct, active definitions of what degree recipients should know and be able to do at each degree level -- remains unchanged," the report says. (A webcast of today's event can be streamed here.)

Given the relatively modest changes in the document itself, the occasion of its release may be best used to assess what impact the Degree Qualifications Profile has (and has not) had since 2011.

What the DQP has not done is clearer: it has not become a tool for policy makers to use to standardize the education provided by colleges around the country, as some critics feared. Nor has it, as Lumina officials probably hoped, been widely used to dramatically reshape the curriculums at many (or perhaps any) colleges.

But what it has done, say officials at numerous colleges, is start or stimulate deeper discussions among faculty members and academic leaders on more than a few campuses about what professors are doing in the classroom, what students should be and are learning, and how the former might be altered to improve the latter.

"On some level, I think the greatest utility of the DQP is to start conversations as opposed to providing a set of benchmarks," says Maria Zack, a professor of mathematics at Point Loma Nazarene University, which used the profile to help gauge the efficacy of the learning that takes place in its capstone experiences for seniors.

What Is It?

The Degree Qualifications Profile is not easy to absorb or understand. The 60-page document that lays out the "completed" version of the profile (as opposed to the "beta" version released in 2011; Lumina officials refuse to call it "final" because it is meant to be forever adapted and refined) offers several different graphical representations (a spider web, a grid, and a matrix) as well as plenty of words to try to make sense of it.

But the gist is that for each of five major learning categories -- specialized knowledge, broad and integrative knowledge, intellectual skills, applied and collaborative learning, and civic and global learning -- it aims to define what proficiencies students should be expected to display at each of the three degree levels. (The "intellectual skills" category has several subsets, including analytic inquiry, use of information resources, engaging diverse perspectives, ethical reasoning, quantitative fluency, and communicative fluency.)

The descriptions of the skills and knowledge are hard to argue with, but can also seem vague -- and probably unthreatening. What might be threatening -- and is certainly ambitious -- is this statement from the document: "The intermediate goal of the DQP process is consensus on a public definition of quality in U.S. higher education." And the long-term goal of the profile and other efforts, the document states, is "to increase the capacity of postsecondary education to ensure that students achieve the levels of learning they require and deserve."

That is especially necessary, Lumina officials argue, given the increasing importance of higher education to individual and societal success, and the growing pressure on colleges and universities to prove that their students are learning.

"The DQP responds to these concerns by describing concretely what is meant by each of the degrees addressed. Focusing on broad areas of conceptual knowledge and essential proficiencies and their applications, the DQP illustrates how students should be expected to perform at progressively more challenging levels. Demonstrated performance at these ascending levels becomes the basis on which students are then awarded degrees."

The framers of the DQP clearly anticipate that language like that will set off those likely to dismiss the degree profile as a tool of the accountability movement and an effort to craft a national definition of higher education quality (that might lend itself to a national test of such quality).

"While clarity and consensus are goals of the DQP process," they write, "the DQP does not attempt to 'standardize' U.S. degrees. The DQP recognizes the role and responsibility of faculty to determine both the content appropriate to different areas of study and the best ways to teach that content."

And to those who might object that the degree outline ignores traits that many might consider essential outcomes of a college education, the DQP authors say: "[T]he emphasis of the DQP on assessable learning is not meant to imply that cognitive standards are sufficient to measure all desirable forms of student development. The DQP chooses not to define 'affective' goals of learning that many colleges properly affirm -- e.g., integrity, personal initiative, professionalism -- because they rarely are specified as criteria for awarding degrees. But the DQP recognizes the value of such goals and encourages institutions to articulate and foster them."

Merisotis, the Lumina president, said in an interview that he believes many of the faculty members and academic leaders who delve deeply into the DQP come to appreciate it.

"When people hear about it, they think standardization," Merisotis said. "But when they read it, and really dig into it, it's different from what they hear."

Outside Views

That wasn't the case for some faculty members Inside Higher Ed asked to review the new version of the DQP.

Peter C. Herman, a professor of English and comparative literature at San Diego State University who harshly criticized the original iteration of the DQP, said he appreciated that in the new version the authors "go out of their way to include faculty, and to couch their proposals in the least aggressive language possible." 

But he said the profile continues to suffer from its push for "homogenization.... It really proposes a specific kind of study aimed at producing a specific result for all institutions. These proposals assume that there is only one end for higher education: preparation for the workforce."

Christopher Newfield, professor of literature and American studies at the University of California at Santa Barbara, said he feared the DQP could become "another bureaucratic compliance language layered on top of existing assessment regimes," and criticized the report for failing to recognize that the primary impediment to better learning at many institutions is underfunding.

"The real project is to scale up existing quality with new technology but especially with new funding, not to reinvent the wheel of good teaching in the form of an abstract benchmarking regime," Newfield said via email. "It would be great if Lumina would send personnel directly to a socioeconomic range of departments and faculty to help develop model programs on the ground, rather than sending more checklists to senior managers."

How It Has Been Used

Scores of colleges and universities have experimented with the DQP in the last three years, and faculty members and academic leaders at several of the institutions said they found the experience useful -- if not transformational.

Norman Jones, professor of history and chair of general education at Utah State University, said that in his state, the DQP was useful in broadening the discussion about a college education from a focus on the major (which tended to focus on employability) to attention to the "wholeness of the degree."

The degree profile also bolstered the role of faculty at Utah State, he said, because "when we started to talk about a degree in terms of competency or proficiency, you get down to this question of how do you know a student has these things. And that you can only answer by going to the faculty," he said.

Scott F. Oates, director of assessment and institutional effectiveness at Virginia Commonwealth University, described how his institution and J. Sargeant Reynolds Community College sought to align the assignments in courses frequently taken by transfer students to smooth their pathway. The DQP provided a "lingua franca" that gave the faculty members at the two institutions a "shared understanding" of what they were striving toward, he said.

But the degree profile was more helpful as a reason to bring faculty together than it was as the ultimate definer of what the professors thought was important, he said.

As part of the Association of American Colleges and Universities' Quality Collaboratives effort, administrators at the institutions thought they might use the DQP to help align the courses, Oates said. But when they brought the instructors from the two institutions together, "they looked at it and said, 'If you let us sit down and work on [the alignment] side by side, we don’t need the DQP.'

"What faculty valued was the time for us to do fine-grained alignment of assignments. When you provide leadership and the kind of places where faculty can talk about assignments, give them tools where they can have higher level vision than just ‘my course,’ they are receptive.… The DQP wound up being interesting but not necessary."

Zack, the math professor at Point Loma Nazarene, said she was suspicious of the degree profile when she first learned of it. "A lot of the conversation in my academic circles was, 'This is going to be the government's path to a set of benchmarks to hold institutions to,' " she said.

But that's not what the DQP is about, Zack said. "You cannot benchmark across institutions with something that can be adapted differently by every institution."

She described herself as "not particularly interested in assessment for assessment’s sake," but rather "interested in assessment for purposes of telling me how the students are doing and how the curriculum might need to be adjusted so they might do better" -- a characterization that would probably apply to many professors.

Lumina's degree profile has "been a good tool for conversation" to get instructors at Point Loma Nazarene thinking collectively about the parts of a higher education that are most important to students at that institution, and for "helping us think about how our students learn."

She adds: "If this turns into a list of specific items that you need to have in the accreditation process or a government accountability system, it will become much less fabulous very quickly."

What's Ahead

George D. Kuh, Chancellor's Professor Emeritus at Indiana University and director of the National Institute for Learning Outcomes Assessment, has led NILOA's three-year review of the DQP. He said there was lots of talk when the degree profile was launched that colleges were going to "reinvent [their] curriculum" around the DQP.

No institution has done so -- that's "too hard to do," he said.

But some colleges have used the profile to better articulate their gen ed curriculums, he said, and others have changed the sequence of their courses to close gaps that they discovered when they used the DQP as a "systematic way of looking at whether what you are teaching and what students are learning are connected."

Kuh said that the real promise of the degree profile is the extent to which professors within and across institutions have a common language and framework on quality student learning. NILOA has created an online library of assignments that are linked to DQP outcomes, and it will establish a team of faculty and staff coaches who will work with other colleges that want to use the DQP and its sister tuning process.

"What's important is what the DQP stimulates on campus," he said, "not adopting it wholesale."