Spiritual Accountability

Susie, like her older brother and sister, enrolled at a Christian university in her home state. While at the university, she became increasingly involved in issues of social justice, helping out regularly at a soup kitchen and organizing a food drive. She attended chapel weekly, as required, but reported that she frequently daydreamed or took it as 'down time' from her busy schedule. Has Susie grown spiritually in college? To what degree? (Please round to the nearest whole number).

It’s a task few assessment experts would envy -- a mandate to measure a student’s spiritual growth in a world far more complex than that simplistically sketched above. How to make the seemingly subjective experience of faith objective, to measure a college student’s spiritual growth as you would a child’s height, with penciled marks noting an inch here, an inch there, on a four-foot paper ruler taped to the president's door?

For many religiously affiliated institutions, that’s not a hypothetical question. As the accountability pressures on higher education grow, and words like “measurable outcomes” become common parlance in academe, religious colleges are increasingly embracing a need to measure the spiritual and moral outcomes they promise in their mission statements to deliver. They're seeking ways not only to measure their own students' spiritual commitments -- and how those commitments might change from freshman to senior year -- but also how they as institutions stack up, spiritually speaking, relative to peer colleges.
    
“We have developed a very strong emphasis on assessment as part of our accreditation process,” says Randall Bell, associate director for the Association for Biblical Higher Education. “What we tell schools is that you are supposed to articulate your intentions and then examine the results of your activities to see if they’re commensurate with your intentions. Most of our schools intend to help our students grow spiritually, so if that’s one of their intentions, they’re looking for ways to assess if that’s in fact happening or not,” he says.

“We wanted to be in some position where we could begin to help make the case with something more than just anecdotal evidence that what we say is integral to the educational experience on one of our campuses is actually taking place,” adds Ronald Mahurin, vice president for professional development and research at the Council for Christian Colleges & Universities. “If we say we have a unique mission and identity, are there reasonable ways in which we can help explain this to a broader public -- not only what we do, but how effective we are in doing this?”

The leaders of public and private institutions alike are thinking about spirituality these days, as the data suggest that's what their students are thinking about, too. Researchers at the Higher Education Research Institute at the University of California at Los Angeles released data in 2005 suggesting that freshmen care about spiritual matters far more than was widely believed -- and that they find guidance from colleges sorely lacking in this domain. The researchers are surveying those same students as juniors this spring to pinpoint any changes in their spiritual lives and the experiences that may have brought these changes about.

“We feel very strongly that ignoring the aspect of spiritual development is ignoring the whole student, especially when we learn from our data that our students are very interested in that,” says Helen Astin, one of the principal researchers for the Spirituality in Higher Education survey. Researchers hope to survey 50,000 students this spring at 150 institutions across the various sectors of higher education, says Astin, who stresses that tapping into students’ spiritual lives is an avenue for enhancing student engagement more generally.

But a key difference is that while Astin says she hopes all types of institutions use the UCLA data to inform their attention to students’ spiritual lives, the publics and non-religious privates generally have the luxury of taking the data or leaving it. More and more, religious institutions -- pressured to provide outcomes data on their accreditation reports relative to “mission” -- don’t see themselves as having that luxury (or, in many cases, even wanting it).

As such, the past five years have been a time of tremendous growth in “spiritual assessment” efforts tailored for students at religious colleges. Some of the efforts are primarily research-oriented, while others are explicitly intended to be used for accountability purposes, as well as to inform decision-making and identify best practices.

“The idea is that we’re really not trying to measure doctrinal beliefs; there are other measures to do that," says Todd Hall, an associate professor of psychology at Biola University, a Christian institution in California, and founder of Concentus Assessment Solutions, a company that offers a Web-based, 170-item “Spiritual Transformation Inventory” designed to quantitatively measure spiritual vitality. "It’s a little easier to measure that, just to ask students what they believe. What’s been difficult, and what we’re trying to measure, is spiritual and character development.”

“We’re trying," Hall adds, "to tap into the gut-level measures of someone’s experience with God.”

The Measures

But what size tap do you use? Where do you drill? How deep? It's not so simple as solely asking if a student prays -- but also how often, what type of prayer and, the big question, the meaning derived by the student through prayer, as Hall explains.

Many of the current efforts to assess spiritual growth in college students evolved, to various degrees, out of the Faithful Change Project, an early effort to patiently measure the steps in a student’s spiritual growth from year to year.

The ongoing Faithful Change research initiative tracked a cohort of students at six Christian colleges from freshman year through their graduation in 2002, relying both on quantitative surveys  and qualitative interviews that are still being coded, well, quantitatively. Researchers attempted to measure the students' spiritual growth across the stages of faith development identified by James Fowler, a theologian and developmental psychologist who recently retired from his position leading Emory University’s Center for Ethics.

“It’s not so much the belief statements that are made, not so much the content, or what might be considered doctrine, as it is the cognitive structure,” says Arthur Nonneman, the chair of the psychology department at Kentucky’s Asbury College. Nonneman is currently completing the coding of the interviews along with Gay Holcomb, an assistant professor of psychology and director of institutional research and assessment at Asbury, a Christian institution. In the interviews, Nonneman and Holcomb looked for evidence of critical thinking: Is a student's faith "borrowed" from parents or peers, or has a student's faith been analyzed, questioned, fought with, owned? 

Their main finding after completing coding of interviews for the students’ first three years (progress on the initiative slowed after a grant from the John Templeton Foundation was largely exhausted) is that crises foster spiritual growth -- not just emotional crises, but also significant intellectual challenges in which students are exposed to diverse ways of thinking through classroom work or multicultural experiences.

“When you’re around people who think differently, you wind up getting challenged. You have to defend yourself and in the process of doing that, you start examining your own beliefs,” says Holcomb.

Out of this research-oriented Faithful Change initiative, Todd Hall, formerly involved with the project, spun off to create a more time-efficient, purely quantitative spiritual assessment.

Hall’s Spiritual Transformation Inventory attempts to assess the strength of a student's relationship with God. Asked to indicate their level of agreement with various statements (from strongly disagree to strongly agree), students select their reactions to prompts like, “I come to know God more fully through my own suffering,” “There is a least one person who is a spiritual mentor in my life,” or “I have friendships in which we regularly challenge each other on our spiritual growth.” About 20 student-specific questions ask about the role that institution-sponsored activities, such as chapel service and ministry sessions, play in fostering the divine relationship: “What impact have mentoring relationships with faculty at your school had on your overall spiritual development?” “What impact have praise and worship sessions sponsored by your school had on your spiritual development?”

Hall's questions, tailored for Christian schools, are rooted in emotion, as opposed to the more cognitive structure embraced by the Faithful Change Project. To what degree does a student have an awareness of God's presence? To what degree, Hall asks, does the student participate in a spiritual community, and feel a sense of belonging to it? How securely does a student experience that relationship with God? 

About 25 institutions affiliated with the Council for Christian College & Universities participated in the Spiritual Transformation Inventory in the 2005-6 school year, with 15 to 20 participating this past semester, Hall says. A smaller-scale collaboration with the Association for Biblical Higher Education began this fall, and Hall is also launching a wide-scale marketing effort to reach Christian high schools. Not only do students get individual score reports deciphering their results, institutions also get group reports that show how their students’ scores stand relative to those reported at other Christian and biblical institutions . “Essentially, what we’re doing is developing national norms, so the schools have a benchmark to compare themselves against,” says Hall.  “It’s designed to provide some fodder for reflection," to encourage institutions to notice their "gaps" and potentially take steps to address areas where they fall below average, he explains.

Meanwhile, many Roman Catholic colleges are engaged in obtaining similar data to assess institutional performance. For instance, Ellen Boylan, director of institutional research and assessment at Marywood University, in Pennsylvania, has designed a set of 20 questions that 36 Catholic institutions are appending to this spring’s administration of the National Survey of Student Engagement.

Students are asked to agree or disagree, on a five-point scale, with prompts like, “The mission of this institution is widely understood by students,” “The heritage of the founding religious community of the institution is evident here,” and “The faculty at this institution discuss the ethical implications of what is being studied.” Other prompts tap into social values: "The faculty, staff and students here are respectful of people of different religions," "The environment here encourages students to develop an appreciation of diversity," and "This institution offers opportunities for volunteering and community service."

On an institutional level, Boylan will be able to measure the changing attitudes of Marywood seniors, who first answered her survey questions as freshmen. More broadly, Boylan, who recently received a Teagle Foundation grant for the research, also hopes to find some non-Catholic colleges to participate – the survey is not faith-specific – to provide some comparison data across sectors.

Other researchers at Catholic institutions have adapted the survey on spirituality created by UCLA researchers, says Jim Trainer, director of planning and assessment at Villanova University and a member of the Catholic Higher Education Research Cooperative, a group of institutional researchers. There’s been a conscious effort among Catholic colleges, Trainer says, to conserve resources and “piggyback on the efforts that are already in place.”

Meanwhile, a survey of alumni from a broad spectrum of institutions conducted by a Minnesota-based higher education management consultant firm, Hardwick~Day, for the National Catholic College Admission Association, attempts to answer the question of how the institutions impact spiritual growth through the perspective of graduates. Among the prompts: Whether the college experience helped in integrating faith with other aspects of life and how effective the college was in fostering the development of a sense of purpose in life.

James Day, Hardwick~Day's principal and founder, says the firm has conducted similar studies commissioned by groups such as the Lutheran Educational Conference of North America and The Annapolis Group, an organization of independent liberal arts colleges. Day says that one benefit of the alumni survey (which focuses on a variety of domains, not just spirituality) is that it allows institutions to gain broader context regarding their successes and weaknesses across the various sectors of higher education.

For example, in the 2004 survey commissioned by the Lutheran association, 58 percent of Lutheran college graduates indicated that they learned more about faith in college, compared to 18 percent of alumni of public flagship universities. In the Catholic colleges survey, conducted this fall and scheduled to be presented Sunday at the annual meeting of the Association of Catholic Colleges and Universities, 80 percent of Catholic college alumni respondents indicated the effectiveness of their college in developing moral principles that can guide actions, compared to 79 percent of respondents from other religious institutions, 56 percent of alumni from non-sectarian private colleges and 35 percent of alumni from flagship publics.

“We’d be disingenuous to say that the market didn’t impact it at all," Trainer of Villanova says about the emphasis on "spiritual accountability." Especially in Catholic higher education, where broader debates are happening about how institutions will remain vital as the number of individuals in religious life declines, this type of assessment may be needed not only for accreditation, but also to satisfy a different hunger for accountability -- to help make the case to the broader public that religious institutions offer something distinctive, says Trainer. "How is it," he asks, "that we carry on the catechism of the congregation of X, Y or Z, as the membership of that congregation decreases?"