A Small Step Toward Transparency

As public pressure has mounted on colleges and universities to measure and report on the academic performance of their students, various experiments have emerged. Groups of colleges, for instance, are developing their own systems for making public information about student academic achievement, such as the Voluntary System of Accountability crafted by the National Association of State Universities and Land-Grant Colleges and the American Association of State Colleges and Universities and the University and College Accountability Network created by the National Association of Independent Colleges and Universities.

And the nudging from Education Secretary Margaret Spellings and her Commission on the Future of Higher Education also helped prod those behind the National Survey of Student Engagement, for the first time, to encourage participants in the annual study to make their results public to a national audience. Through an arrangement with USA Today, announced last summer, the sponsors of the survey of students' academic involvement and engagement (commonly known as NSSE, or "Nessie") invited all colleges that participated in the survey from 2005 through 2007 to include their summary statistics in a database to be published on the national newspaper's Web site Monday.

“Schools that do participate in this initiative will be able to declare and demonstrate their commitment to improving and being accountable for undergraduate education,” organizers of NSSE said in an FAQ at the time they announced the partnership with USA Today. The group’s board believes that “the time has come for participating institutions to stand together in promoting responsible ways to make available information about important, relevant features of institutional and student performance, and to continue to provide leadership for improving the quality of undergraduate education.”

Monday, upon the annual release of the 2007 scores on the engagement survey, USA Today published its database of benchmark scores on NSSE, and it contained 257 colleges and universities – about a quarter of the 1,000 or so institutions that George D. Kuh, NSSE’s director and a professor of higher education at Indiana University at Bloomington., had invited to participate. (A list of the participating institutions appears at the bottom of this article.)

Kuh said that a majority of colleges never responded to NSSE’s invitations this summer to participate in the USA Today experiment, and that his informal inquiries found that the invitations appeared to have fallen through the cracks at a sizable portion of institutions, never quite making it to the officials best positioned to make a decision to participate. “Of the presidents and provosts I called after the fact, all but one said, ‘We would have done this if we had known about it,’“ Kuh said.

Reasons Not to Participate

Kuh acknowledged that some institutions had expressed concerns about whether USA Today would present the information in a responsible and appropriate way, avoiding the sorts of rankings of institutions that can badly oversimplify complex and easy-to-misinterpret data. Some colleges seemed to want to wait to see how the newspaper handled it -- and to its credit, Kuh said, USA Today "made the results accessible but also went to great lengths to explain it and put it in context. I think they've done a public service."

Inside Higher Ed’s own inquiries to the non-participating colleges found a mix of reasons for their decisions. Pedro Reyes, associate vice chancellor for academic planning and assessment at the University of Texas System – all of whose nine campuses participate in NSSE, but none of which joined the USA Today effort -- said officials there never learned of the requests. He noted that all UT campuses publish their NSSE scores annually (see page 48) as part of the Texas system’s own accountability system. “We certainly don’t have any problems sharing this information,” Reyes said.

Barbara Daus, special assistant to the chancellor at the University of Wisconsin at Platteville, said she suspected the invitation on her campus got lost in the shuffle during a transition in its one-person institutional research office.

Other campuses, though, made conscious decisions not to participate in the NSSE- USA Today collaboration.

Debra B. Jackson, associate provost and assistant to the president at Clemson University, said the university expected to post its NSSE scores by early next year on the Web site it is developing as part of the land-grant university association’s Voluntary System of Accountability. “We’re going to be presenting it there in a way that we think will be of value to students and particularly to parents, in the context of lots of other information that they care about, such as what majors students are enrolling in, our graduation and retention rates, and class sizes,” Jackson said. “Like everything else, NSSE needs to be put in perspective – it’s one measure of one essence of being in college.

She added: “We’re excited about the opportunity to make it public -- our numbers are really good.”

Some institutions said they had little interest in making the data public. Phillip Brown, director of institutional research and studies at Southern Illinois University at Edwardsville, said via e-mail that his institution has participated in NSSE since 2000, but that it views the survey “as an internal assessment tool.... It was designed and developed as an internal assessment tool. It was not meant as a comparative measure between schools.”

Brown added: “Student engagement is one indicator of the college experience that needs to be evaluated among a host of others before deciding on a particular school. Deciding on which college to attend is a complex decision. As much as some seem to wish it, it can’t be reduced to a score on a piece of paper.”

Reasons for Joining In

The colleges that agreed to participate cover the gamut of American higher education – small liberal arts colleges, large public research universities, and everything in between. The trait that links most of them, perhaps not surprisingly, is that the vast majority of them scored well on the engagement survey: 80 percent of the 257 participants posted above average scores for their type of institution in at least half of the 10 categories (five for first-year students and five for seniors). Lake Wobegon comes to higher education.

But not all of the participating institutions scored consistently highly on the survey. The University of Michigan’s Flint campus scored below average for medium-sized master’s level colleges and universities in 9 of the 10 categories presented -- not surprising, Kuh said, given the fact that commuter campuses that educate large numbers of adults, like Flint, tend to fare less well on student engagement measures. But Jennifer Hogan, university relations director at the university, said it was proud of the strong relationships that students reported having with faculty members, and that the institution’s plans to add a residence hall and to build out a fledgling “learning communities” program should boost its scores in coming years.

“We want people to know about the good things we’re doing, and even in some of the areas that are listed as lower performing, we think any good institution should look at itself critically and say, ‘This is where we need to improve and this is what we need to do to get there,’ “ Hogan said. “Why not be open about it?”

Texas Tech University is, like Clemson, an active participant in the NASULCG Voluntary System of Accountability, which also incorporates scores on the student engagement survey. But Michael Shonrock, vice president for student affairs at the Lubbock, Tex., research institution, said its officials thought it better to share its NSSE scores as widely as possible, even though it scored below the average for its peer group in more categories than not. "It's helpful for us to take a snapshot of where we are, and we see absolutely no reason why we would not be more transparent and provide more information to prospective students and parents, and to state and federal officials," Shonrock said. When the invitation from NSSE came, he said, the response from the president's cabinet "was a resounding, Let's do it."

Berry College, in Georgia, scored below its peer private, baccalaureate arts and sciences colleges on 7 of the 10 benchmark scores included in USA Today’s report, though its students reported above average scores on having a supportive campus environment and seniors reported above average scores on student-faculty interaction.

Berry’s president, Stephen R. Briggs, said that the college scored extraordinarily high on some NSSE questions that don’t show up in the summary data presented by USA Today – virtually all Berry students work on the campus as part of the curriculum, in ways that help them prepare for professional life after college, for instance.

But Briggs said Berry decided to participate because he believed the NSSE/ USA Today collaboration was one potentially useful alternative to the U.S. News & World Report rankings that Briggs and many other liberal arts college presidents have been campaigning against. “There’s been a lot of criticism, but we really haven’t offered an alternative,” Briggs said. “Here we had a reasonable alternative, and sure, we wish we had looked a little better. But this helps us understand what’s going on [on our campus] and helps our students and our prospective students understand what’s really going on.”

He added: “If we as an institution say that we should be willing to make ourselves accountable to the public, we need to do that seriously.”

• Adrian College
• Agnes Scott College
• Alaska Pacific University
• Alcorn State University
• Allegheny College
• Alverno College
• Angelo State University
• Appalachian State University
• Arkansas Tech University
• Auburn University- Main Campus
• Augustana College
• Baker University
• Baldwin-Wallace College
• Bard College at Simon's Rock
• Barton College
• Baylor University
• Bellarmine University
• Belmont Abbey College
• Beloit College
• Benedictine College
• Bennington College
• Berea College
• Berry College
• Bethel University
• Brigham Young University-Idaho
• Butler University
• California Lutheran University
• California State University-Dominguez Hills
• Calvin College
• Campbellsville University
• Centre College
• Chaminade University of Honolulu
• Chatham University
• Chestnut Hill College
• Coker College
• College of Notre Dame of Maryland
• College of Saint Mary
• College of Saint Scholastica
• College of the Atlantic
• Columbus College of Art and Design
• Concord University
• Concordia University-Saint Paul
• Converse College
• Covenant College
• Daemen College
• DePaul University
• DePauw University
• Dickinson College
• Dominican University
• Drake University
• Drury University
• Earlham College
• Eastern Kentucky University
• Ecclesia College
• Eckerd College
• Elizabethtown College
• Elon University
• Eureka College
• Evergreen State College
• Faulkner University
• Fayetteville State University
• Fisk University
• Florida Atlantic University
• Franklin and Marshall College
• Franklin W. Olin College of Engineering
• Frostburg State University
• Furman University
• Gardner-Webb University
• George Mason University
• Georgia State University
• Graceland University-Lamoni
• Grand View College
• Greensboro College
• Grinnell College
• Gwynedd Mercy College
• Hamline University
• Hampden-Sydney College
• Hampshire College
• Hanover College
• Harvey Mudd College
• Hastings College
• Hendrix College
• Indiana University-Bloomington
• Indiana University-East
• Indiana University-Kokomo
• Indiana University-South Bend
• Jacksonville State University
• James Madison University
• John Brown University
• Judson College (Ala.)
• Judson University
• Juniata College
• Kalamazoo College
• Kansas State University
• Keuka College
• Knox College
• La Salle University
• LaGrange College
• Laguna College of Art and Design
• Lindsey Wilson College
• Linfield College
• Lock Haven University of Pennsylvania
• Longwood University
• Loyola College in Maryland
• Luther College
• Lynchburg College
• Malone College
• Marlboro College
• Mars Hill College
• Mary Baldwin College
• Maryville College
• Mercer University
• Meredith College
• Messiah College
• Miami University-Oxford
• Michigan Technological University
• Middlebury College
• Milligan College
• Millikin University
• Milwaukee School of Engineering
• Misericordia University
• Missouri Valley College
• Monmouth College
• Montana State University-Bozeman
• Morningside College
• Mount Marty College
• Mount Mary College
• Mountain State University
• National University
• Nebraska Methodist College of Nursing & Allied Health
• Nevada State College at Henderson
• New Jersey Institute of Technology
• Niagara University
• Northern Arizona University
• Norwich University
• Notre Dame de Namur University
• Ohio University-Main Campus
• Oklahoma City University
• Oklahoma State University-Main Campus
• Oral Roberts University
• Ouachita Baptist University
• Our Lady of the Lake University
• Pace University
• Palm Beach Atlantic University
• Peace College
• Pennsylvania College of Technology
• Pepperdine University
• Polytechnic University
• Purdue University-Main Campus
• Randolph-Macon College
• Rhodes College
• Rockford College
• Rollins College
• Rose-Hulman Institute of Technolog
• Saint Ambrose University
• St. Francis College (N.Y.)
• St. John's University (N.Y.)
• Saint Joseph College
• Saint Mary's College (Ind.)
• St. Mary's University (Tex.)
• Saint Mary's University of Minnesota
• Saint Michael's College
• Saint Xavier University
• Santa Clara University
• Scripps College
• Seattle Pacific University
• Seattle University
• Slippery Rock University of Pennsylvania
• Sonoma State University
• Southeast Missouri State University
• Southeastern Bible College
• Southern Catholic College
• Southern Nazarene University
• Southern Virginia University
• Southwestern University
• Spring Hill College
• Springfield College
• SUNY College at Buffalo
• SUNY College of Environmental Science and Forestry
• Susquehanna University
• Sweet Briar College
• Temple University
• Texas A&M University
• Texas Tech University
• Texas Woman's University
• Tusculum College
• Union College
• United States Military Academy
• Universidad Politecnica de Puerto Rico
• University of Arkansas-Fort Smith
• University of California-Merced
• University of Central Arkansas
• University of Charleston
• University of Cincinnati-Main Campus
• University of Colorado at Boulder
• University of Dayton
• University of Hartford
• University of Kansas-Main Campus
• University of Maine at Farmington
• University of Michigan at Ann Arbor
• University of Michigan at Flint
• University of Minnesota Crookston
• University of Missouri-Kansas City
• University of Missouri at St. Louis
• University of Nebraska at Kearney
• University of Nebraska at Lincoln
• University of New England
• University of North Carolina- Asheville
• University of North Carolina-Wilmington
• University of Pittsburgh-Bradford
• University of San Francisco
• University of Science and Arts of Oklahoma
• University of South Carolina-Aiken
• University of Southern Mississippi
• University of St Francis
• University of St. Thomas (Minn.)
• University of St. Thomas (Texas)
• University of the Ozarks
• University of Virginia
• University of Wisconsin-Eau Claire
• University of Wisconsin-La Crosse
• University of Wisconsin-Parkside
• University of Wisconsin-Stevens Point
• University of Wyoming
• Urbana University
• Ursinus College
• Ursuline College
• Valdosta State University
• Virginia Wesleyan College
• Viterbo University
• Wabash College
• Wagner College
• Walsh University
• Warner Southern College
• Wartburg College
• Washington & Jefferson College
• Wentworth Institute of Technology
• Wesleyan College
• West Texas A&M University
• Western Carolina University
• Western Illinois University
• Western Kentucky University
• Western Michigan University
• Westminster College
• Westminster College (Utah)
• Wheaton College (Ill.)
• Wheaton College (Mass.)
• Whittier College
• Widener University
• William Woods University
• Wilmington College
• Wingate University
• Winthrop University
• Wittenberg University
• Wofford College
• Worcester Polytechnic Institute
• Xavier University (Ohio)