Mapping Student Learning With Precision

West Point knows what it wants.

And, while many colleges come under criticism on this next detail, the United States Military Academy at West Point also knows how it wants students to get there. Which is anywhere.

"Our graduates are likely to be stationed all over the world. They're going to confront challenges that quite frankly we couldn't have prepared them for. We couldn't have known what the challenges would be," said Bruce Keith, a professor of sociology and associate dean for academic affairs. "We have to develop in them a foundation for lifelong learning."

With broad goals of lifelong learning and liberal education at the fore, West Point professors are diligently pursuing an ongoing effort to map where, in what classes, and how students achieve the outcomes West Point wants. Yet, even considering West Point's heavy core curriculum -- which consists of about 30 courses, Keith said, 26 of which are taken in common by all students -- the circled route consists of many branching roads that ultimately, faculty hope, converge.

And while West Point is of course a unique institution, some suggest it could be a model for how colleges can become more intentional in linking all of these branches of the undergraduate experience.

A Multi-Layered System

At the most macro level, West Point has identified six curricular domains: The intellectual, military, physical, moral-ethical, and social domains, as well as the “domain of the human spirit.”

Drilling down, "We have, over the last couple of years, tried to identify the goals and outcomes associated with each of those domains," Keith said. Each of the domains includes a series of goals. In the intellectual domain, there are nine. (Among them: “Graduates apply mathematics, science, technology and the engineering design process to devise technological problem solutions that are effective and adaptable," and “Graduates draw from an appreciation of culture to understand in a global context human behavior, achievement and ideas.” Or, here’s the most concise one: “Graduates think and act creatively.”)

For each intellectual goal, there is a "faculty goal team," charged with defining the goal and measuring student progress against it.

Drilling down further, within each goal, the faculty team determines the goal's dimensions.

For the engineering and technology goal – which, like all goals, all students are expected to meet regardless of major, and which non-majors satisfy through a required three-course engineering sequence -- a faculty team has mapped out a set of 12 expectations, or dimensions, that align with ABET's accreditation standards (ABET is a specialized accreditor focused primarily on engineering and technology programs). Meanwhile, for creativity, “We took that goal and broke it down into three dimensions,” said Marie Johnson, a professor of geology and that goal team's chair. It’s a systematic process, defining and measuring creativity, which the military academy has defined as “novelty that is useful.”

To demonstrate creative thinking and acting, students are expected to be able to develop multiple alternatives to address a given problem, and decide amongst them, and to effectively transfer knowledge between disciplines (rather than, say, take a creative writing or painting class). For assessment purposes, the goal team has developed 13 statements describing behaviors that correlate with the creativity goal and its dimensions: For instance, creative students, by West Point’s standards, “question assumptions” and “respond to ambiguity with persistence.” Using a 1-5 Likert scale, the faculty goal team has asked professors to score students in their "integrative experience," or capstone-like courses, on the degree to which they display each of 13 behaviors associated with each of three dimensions associated with the goal of creativity, one of nine goals in the intellectual domain, one of six domains (the other five, Keith said, are not as well-developed). Whew.

“We spent the better part of 12 years working through this process and we have a very well-defined system," Keith said.

'Curricular Intent'

Sound daunting?

While West Point is uniquely positioned to tackle such a cartographic task, given its extensive core curriculum and the motivated students it attracts, its example is nevertheless instructive elsewhere, suggested Carol Geary Schneider, president of the Association of American Colleges and Universities. Schneider and Keith both described congruence between West Point's designated learning outcomes and those identified in AAC&U's Liberal Education and America's Promise (LEAP) project, which stresses a need to broadly educate students so they can learn and adapt as the world, and career paths, change. ("We can't have people who are trying to live by the rule book in an unscripted world," Schneider said.)

"They're asking, really, how and where the different strands in a student's education come together in an integrated whole," said Schneider. "Many AAC&U members are saying that if we want our students to be more successful as learners, we have to be more intentional as institutions and as educators, meaning we have to know better what it is we want to help them achieve and where it is they're working on different goals. That's a broad mandate that many of our members have adopted. But when I look at West Point or the Air Force Academy [which has done similar work], I see the full meaning of this mandate."

"I think both West Point and some of the other military academies help us see what collaborative faculty and staff leadership could look like."

Keith explained that West Point's own work on this front first began around 1989 when the Middle States Commission on Higher Education challenged the academy on the coherence of its curricular framework -- why these courses versus those -- and assessment. "It was an issue of curricular intent and assessment to inform curricular change. And we couldn't answer either one of those questions."

"What may work at one institution doesn't necessarily work at another institution because of different cultures, different perspectives, different missions," said Keith. "But what does work is this process of asking faculty amongst themselves about this notion of an educated graduate, and to demonstrate that they're actually achieving this."

Assessment

When it comes to the engineering goal, for example, Col. Barry Shoop, deputy head of the department of electrical engineering and computer science, and the goal team chair, said the team members have been collecting data for five years now, mainly by relying on graded indicators like tests and presentations already "embedded" in the engineering classes. ("In order to have an effective assessment process, it has to be efficient. Otherwise, you end up assessing your brains out. You have so much data lying around you never end up evaluating or coming to conclusions on it," he said).

Now, five years in, some patterns are emerging, including one area of concern: Weak achievement among non-engineering majors on the goal's 12th and final dimension, "In response to a technological problem, learn new concepts in engineering and learn new technologies without the aid of formal instruction."

"One of my goals for this next semester is to look longitudinally now at the data and see whether we're asking too much from the three-course engineering sequence," said Colonel Shoop, noting that the academy used to require a five-course engineering sequence of all graduates.

Meanwhile, moving beyond engineering, Colonel Shoop also sits on a steering committee, comprised of the chairs of all the goal teams, where faculty can discuss what they're learning at yet another level (imagine here a topographic map).

"We share best practices at that level," he explained. "So if somebody finds a better way of doing something in, say, the math or science goal, or the engineering and technology goal, and there's a way to translate that to another goal team to make it more efficient, we do that."

"Being a military organization," Colonel Shoop acknowledged, "helps too."