Wednesday
October 15, 2008
Advertisement
Advertisement
News, Views and Careers for All of Higher Education
Nov. 20, 2007
Another Call for Assessment
As college associations unveil new ways for colleges to report on what they do, and Congress debates how much accreditors should ask of colleges, an effort has been going on for months to craft a national statement on student learning and assessment.
Related stories
Drafts of the document leaked to Inside Higher Ed, by educators concerned about the statement’s direction, suggest that colleges commit to “gather evidence” about the success of students in meeting certain education goals; that the evidence should be shared widely and used by accreditors; and that in some cases this material should be used to compare institutions. “Such information and evidence allows comparisons, where appropriate, between and among institutions and may suggest areas for improvement,” the draft says. At the same time, the document asserts that the “primary responsibility for achieving excellence” rests with colleges, and not outside bodies.
The effort is being coordinated by the Association of American Colleges & Universities and the Council for Higher Education Accreditation, along with the Teagle Foundation. A final version of the plan is expected to be released to the public in January, at the accrediting group’s annual meeting.
The document takes a more philosophical approach to higher education than some other recent efforts, calling on colleges not only to report on what they accomplish, but to set out to accomplish certain specific things. For instance, the draft says that while goals should vary by institutional mission and student body, goals generally should include “the enrichment of both our democratic society and individual lives through the study of science, social science, the humanities and the arts.” The rhetoric of the document — while containing references to economic competitiveness and the global economy — also has specific references to the liberal arts in a way that has not always been present in some recent calls for more accountability and assessment.
The aim of the draft was to have numerous college associations sign on to the framework outlined, with the idea of then encouraging their members to join in “a compact” to commit to the document. Some higher education leaders have strongly backed the efforts, arguing that the best way to fend off government intrusion is for academe to set its own standards.
But parts of the document are controversial. While education groups agree that colleges should have goals and that they should consider how to improve the education they offer, many fear that moves to measure student learning will inevitably lead to the use of standardized testing and to facile comparisons of institutions. As a result of such criticisms, the draft is no longer called a “compact,” and plans to have groups endorse it at the time it is released have been dropped.
The American Council of Learned Societies explicitly rejected the effort and an e-mail from Pauline Yu, president of the council, explaining the rationale for doing so, is circulating among some critics of the effort. (Yu could not be reached and was not among those who provided information about the draft statement.) In her memo, Yu called the draft “conceptually flawed” and “rhetorically risky” and expressed fear that it could undo some of the qualities of American higher education — “decentralized, market-driven and pluralistic” — that have led to its success.
Noting that the draft calls for colleges not only to develop and measure standards, but to show how they would use those standards and the measurements to engage in “systematic improvement,” Yu wrote that colleges could be setting themselves up to divert resources and invite more government scrutiny, not less.
“Where will the resources come from for this investment in an entirely new level of bureaucracy that will be devising programs and gathering evidence? Teaching? Research? Paying for a new ETS? Wouldn’t we rather have those energies and funds devoted to strategies that we do know improve student learning, like smaller classes and capstone courses?” Yu wrote.
In addition, Yu said that the draft “explicitly invites more monitoring from the government and, indeed, perhaps unwittingly offers up its process as the yardstick for determining funding and regulation. The annual reporting it proposes runs the risk of fueling the ongoing movement to reduce educational autonomy — more testing, more bureacracy, more captious regulation, more lawsuits.”
Carol Geary Schneider, president of the AAC&U, said that some of the concerns expressed during the process of creating the draft have been addressed. “People will be pleasantly surprised,” she said. Schneider said that the leaked draft didn’t reflect some of those changes, although she declined to release a current version of the draft.
The effort, she stressed, was not focused on tests. “AAC&U is deeply committed to forms of assessment that are anchored in the curriculum, that are supervised by faculty and result in strengthening the quality of learning,” she said. “The document is not about testing. We did not have in mind testing as the primary focus.”
Schneider contrasted the document she has helped prepare with others. “If there is one thing that is fresh to this discussion, it is the call to define our educational purposes first,” she said. “Most of the discussions have been a frantic hunt for the right measure to report scores. This is about a thoughtful quest for the right aims to guide learning.”
While Schneider said she understood the concerns of some critics, she said it was important for higher education to respond to public demands for accountability. “One of the primary missions of all colleges and universities and is the quality of learning. Therefore it is crucially important that colleges and universities themselves be the primary leader in defining aims and creating forms of assessment that are worthy of our mission.” (Schneider’s organization is among three higher education groups that received a $2.4 million Education Department grant to develop and test tools for measuring student learning outcomes.)
Terry W. Hartle, senior vice president for government and public affairs at the American Council on Education, said his group supported this effort — along with other assessment initiatives being started by various academic groups. He said that given the diversity of higher education, it wasn’t surprising that disputes would surface over the best way to proceed. But he said it was important for college organizations to move ahead in developing systems they could support.
“The more activities that are under way, the harder it gets for critics to say colleges and universities aren’t engaged with these issues,” he said.
Want it on paper? Print this page.
Know someone who’d be interested? Forward this story.
Want to stay informed? Sign up for free daily news e-mail.
Advertisement
Comments
Student learning outcomes, without standardized testing
Those who fear that university attention to student learning outcomes will require standardized tests would do well to consult the academic journal On the Horizon, vol. 15, no.2 (Spring 2006). The entire issue is devoted to the theme “Focusing the Institution on Student Learning Outcomes.”
David Shupe, Director at Evaluation of Student Achievement, at 9:10 am EST on November 20, 2007
Correct year for journal
The correct date for the On the Horizon issue is Spring 2007.
David Shupe, at 9:20 am EST on November 20, 2007
Double Standard
If the driving force on assessment is money (i.e. taxpayer monery), we have a clear problem. For example, why should I not have the same sort of assessment clout over, say, military expenditures as the assessment drivers have over education? After all, I pay taxes and military expenditure is how a substantial part of my money is used. I would go so far as to say that I have a much better understanding of things military than most of them have an understanding of higher education. The bottom line seems to be much of this is happening as a knee jerk response to issues decades in the making and because the politicians and activitis have begun making the same flat-comparisons of scores between countries as they claim they will not make between schools.
The strength of these calls may be because higher education is vulnerable. To me it seems likely that much of the push is akin to bullying against a group that has little ability or desire to stand up for itself. As with all bullying, however, one must either stand up or live in fear.
Unimpressed, Dean, at 9:25 am EST on November 20, 2007
agreed
I agree. Why are so many other public sectors free from scrutiny and oversight, like the military, prisons, and even the Boards that lead public institutions? Why does the government find it unnecessary to make these entities prove their worth? After all, the military receives much more in public funding than higher education and many states now devote more funding to prisons than higher education.
When the military has to go through the same assessment and evaluation procedures as higher education, you will see support. Until that unlikely day, back off.
PS, at 10:10 am EST on November 20, 2007
Assessment—on and on and....
I find this to be about pedestrian pursuits focused more on the marketplace than on learning. I am struck by the willingness of the professoriate to permit the intrusion of bureaucratic interests into their knowledge in their field of knowledge and into their delivery of that knowledge into the classroom. The Academy will suffer serious losses if the assessment gurus win this ideological struggle.
We may loose our way entirely. We may end up measuring how many angels can dance on the head of a pin.
It was well and simply said thousands of years ago:The purpose of education is moral development. —Plato
Ollie, at 10:10 am EST on November 20, 2007
Just be happy that a couple of congressmen don’t take a half day tour of your campus in order to decide whether or not to shut your school down and transfer your department to the University of Alaska.
Wesley, at 10:40 am EST on November 20, 2007
As an astute scholar once pointed out...
... the reason people choose to work in higher education is PRECISELY because of the lack of accountability. You can’t expect accountabilty from an enterprise that was built precisely to avoid it.
SB, at 10:55 am EST on November 20, 2007
Institutional assessment is largely misguided
I am sorry but institutional assessment is largely a sham and the biggest resource waste here is devoting time to this issue. College students are adults and if they don’t take advantage of the unbelievably large treasure trove of resources that colleges and universities place at their disposal, why should the university be held accountable? A full-time students spends on average between 12 and 18 hours per week in face-to-face contact with faculty. In fact, most of their learning takes place outside the classroom in the form of homework and self-study through tools like university libraries, computing and peer interactions. Yet NESSIE figures tell us that students spend around 10 hours per week outside the classroom on education-related activities and about half the time the professors expect them to spend. That is pathetic.
It is remarkable that we hold ourselves up as the best higher education system in the world and then simultaneously trash it. By and large, faculty members care about their students and I’d bet that any one of them would tell you that students who put the effort in and show interest are rewarded handsomely by faculty willing to take the time to see that learning takes place. Quite frankly, if a student is going to college simply because society says he/she needs to, and not because they care about getting a degree, it is not the institution’s responsibility to force that student to learn. I have seen countless students sleep through class, cram for exams and then consciously choose to spend the bulk of their “face time” with professors bartering for two or three additional points on their exam. That, again, is pathetic.
Universities are not factories that push knowledge into people’s heads through some automated process; they are resource hubs that those willing to seek higher learning can draw on to enrich themselves. Those who succeed in college recognize this and prosper accordingly. Depressingly, institutional assessments of learning outcomes quite wrongly treat institutions like the former and then penalize institutions for results that are largely beyond their control.
There is nothing wrong with ensuring that the high cost students must pay for a higher education can be justified. At the same time, accreditation and quality assurance should be there to make sure that the metaphorical restaurant’s kitchen is clean or that the food it serves is not contaminated. It should NOT be there to penalize the restaurant because the patrons don’t eat everything on their plate or walk away still feeling hungry. I’m also not saying that faculty members cannot be great motivators nor am I absolving them of their responsibility to push students to shine. I am simply asking that the powers that be stop for just a moment and think about which party is truly more responsible, and hence ought to be more accountable, for the outcomes of adults seeking an education.
The voice of sanity, at 12:30 pm EST on November 20, 2007
Assessment-a bad thing?
While externally mandated assessment may indeed bring expectations and means of measurement that are far from helpful, some of these posts sound like assessment is something to be avoided at all costs. Wouldn’t it ultimately be valuable for us all in academia to know what we’re trying to accomplish and to what extent we’re succeeding? That’s something I work hard at determining in my own classrooms. Assessment is essential for effectiveness and something everyone in higher ed should be involved in.
Cristy Bruns, at 12:30 pm EST on November 20, 2007
on assessing
Like many, I’m not whole heartedly against assessment, but it’s the numbers driven faux “institutional” or “programmatic” assessments that bother me as we try to quantify the unquantifiable. We are undergoing this discussion on our campus right now. The biggest hindrance is that so many students exert minimal effort, as was already pointed out, and because of that, fair poorly. Why should a college or teacher be held accountable for flunking those who deserve to flunk? Pretty soon everywhere is going to be Lake Wobegone, where everybody is above average, or so we think.
bradley bleck, instructor at Spokane Falls CC, at 12:45 pm EST on November 20, 2007
Gradgrind Lives for Assessment
The double standard argument is interesting but probably impotent (no one really cares). What rings true is the argument that bureaucrats have found a fresh and vulnerable, i.e., not well defended, “market” to pursue their interests and to engage in money seeking activities (for them) such as “assessment,” the pursuit of “efficiency” in education, the search for “best practices” and the cultivation of arcane “evidence” and “measures” of educational attainment among undergraduates.
I am intrigued by the notion that what we have here is an “ideological” struggle, where the stakes are high, but the ideologies at stake need to be clarified. A key question one hears faculty members ask at regional and national disciplinary conferences is, “Whose interests are being served by this (witch-)’hunt’ for measures?” “What agendas are being served?” Intellectually, clarification of this ground is fundamental.
My own admittedly limited sense of this ground’s battle lines has already been outlined above: the explosive growth of administrative thought and its agents is a parasite on the true goals of education.
For example, what assessments are being made at Harvard or Princeton? Or, to look into public schools, what assessments are being made at UC Berkeley? Shouldn’t all schools be assessed by those measures? If not, why not?
Of course, James Slevin has made a number of useful arguments in this “assessment” arena, beginning, to my mind, in his book Introducing English. He makes the point that the smaller public colleges and universities, not least of which the most vulnerable with the most vulnerable student populations, community colleges, will take the brunt of all this assessment torture, and the result will be, I would guess, better heeled faculties at those schools and a more homogenized workforce for business. The tragedy is that the parents and students of these smaller public higher education institutions really do see the value of higher education as some sort of gold ticket to a job or vocation or technical degree, rather than, as one poster reminded us, of moral development, perspectival expansion, deepening and broadening of literacy, and the maturation of one’s critical faculties.
So the job of faculty members around the country is to educate the public about the true goals of education, reminding the public that marketplace skills, while a desirable benefit of “producing” a well educated worker, is not the primary social goal of educating that citizen.
I’m always been astonished that we need to more vigorously assess and evaluate what many around the world agree to be an American strength, its colleges and universities, while we forget that our K-12 schools, which have been terribly burdened by assessment culture and administrative growth, have declined or fared poorly internationally in their “performance objectives.”
Swifty, Dr. Swifty at small college, at 1:05 pm EST on November 20, 2007
Bad Comparisons
Those of you arguing that the military should be assessed before higher ed are demonstrating why higher ed needs assessing: poor quality control.
The military is completely government controlled. Its funding is solely determined by the government which determines whether or not outcomes and objectives have been met and funded accordingly.
Soldiers are paid to serve.
Students pay to go to college, or someone pays on their behalf. Often these third party payments are govt funds. Which begs the question of what are the govt and taxpayers getting return?
Mainly arguments and protests: “Oh, go away and test someone else, we’re the experts. Trust us. Give us money. No, more money.”
Tod, at 1:20 pm EST on November 20, 2007
Cynical or realistic?
Assessment as it now stands, has a fundamental flaw. Congress tosses equal amounts of money and accountability snafu’s at education wanting ‘bang for the buck’. We ask schools/colleges/universities to come up with assessment plans to judge their own worthiness. Schools must measure outcomes, create measureable results, and make changes to their individual systems where-ever these so called measurable results seem to point. Imagine a school studying itself for the sake of improvement. Sounds like a grand idea, what’s not to like? Assessment is the same idea as Congress overseeing itself. Has it worked? At what cost to resources? Where has the extra workload been deposited for this assessment/accountability exercise/mandate? Are institutional assessments really geared to see “how are we doing” or patterned to preconceived results? Ask these questions and the answers tell the tale…
Lesson learned: If you don’t want congress to dry up the money supply, then produce results; flawed or not. At least we can get on to the important part of education, which is the classroom where it begins and ends.
Bill, at 1:25 pm EST on November 20, 2007
The dialogue on assessment points up yet another hypocritical asepct of collective higher education. Institutions create a market for their wares by creating a caste system in the employment marketplace. They want baccalaureate and graduate degree holders to be given preference, and they generally get their way because over the years the college graduate stereotype has become more powerful as a discriminator in hiring.
Yet, the current resistance to common assessment measures and accountability reflects an antipathy to even a common core of academic accomplishments for all graduates. Given the breadth of institutional autonomy, there is none, of course, and the situation is almost as ludicrous as asking the public to believe the concept of the student athelete. It is reasonable to assert that some with graduate degrees couldn’t meet the freshman admission requirements at some institutions.
Given such disparities, we should come clean by telling employers to discount degrees as job certification, because we can’t assure they have a common meaning, save perhaps, in a few professional fields.
The unjust disrimination we foster in employment is as insidious as that based on enthic, racial, and sexual stereotypes.
This is not a popular view among the “give us your money, don’t ask questions, and take our word for it” crowd. And the current noble sounding arguments opposing standardized assessment across institutions is a diect reflection of this philosophy.
Surely there a few people with integrity in higher education.
Higher Ed Diogenes
Higther Ed Diogenes, at 2:25 pm EST on November 20, 2007
Proponents of standardized assessment in college seek to take a complex learning experience, in which many more aspects of learning that can be measured are enhanced, and simplify it for ease of digestion. Can you simplify the measure or results without simplifying the learning experience? When you figure that out, we’ll talk.
On a related note, many seem to be directly correlating, perhaps even confusing, assessment of student learning with assessment of faculty performance. Somehow this argument has the tone of the student/parent who feels he/she is paying for a grade. As “a voice of reason” said, we teach adults with free will and no truant officer.
M, Instructor at an Illinois community college, at 7:10 pm EST on November 20, 2007
Diogenes, I saw your lamp
I agree with your comments regarding the hypocrisy of collective higher ed. I would like to add another concerning our obligation to employers. Not only should we “[tell} employers to discount degrees as job certification because we can’t assure they have a common meaning, save perhaps, in a few professional fields,” we should also indicate that we have been so successful at selling the populace on their need for a baccalaureate or graduate degree that many are matriculating who at earlier points would never have been admitted. The construct of degree as commodity pretty much guarantees that if they pay the tuition, they get the degree.
An honest (wo)man
Lulu, at 7:10 pm EST on November 20, 2007
“The voice of sanity” hits the key point: Assessment is largely a sham. It is not clear if good data can be obtained by any method.
I am one of many from the STEM areas who has worked with education colleges and been astounded by what I found: (1) Assessment is approximately data free. (2) Everyone has a wacky theory on how to revolutionize teaching, and they assess their pilot program to show it is perfect. (3) It is regarded as a crime to expect students to actually practice, let alone drill.
This is why we import scientists and engineers from countries without the benefit of education theorists.
Raoul Ohio, at 9:35 pm EST on November 20, 2007
Assessment: Fear of the Unknown and Unimagined?
How do we know what students have actually learned in our classrooms, if we do not assess them through things they produce for us—like papers, projects, research assignments, multimedia presentations and more? How in fact do we know that they have actually learned what we expect they should learn, to claim they have achieved a degree at our institution? How do we know that our curriculum is actually coherent and with a sense of integrity, and not only in the way we have spelled it out on paper, but even more so in what a student actually learns through it?
Institutional attention to student learning that approaches assessment as an add-on process rather than as integral practice is at best disconnected from the heart of good teaching and effective learning. At worst it can be intrusive and oppressive. But this does not make assessment itself necessarily objectionable; it is how it is being framed, understood and used that makes it either an important element of quality teaching and curricular development, or an administrative oppression.
A key problem in understanding the integral value of assessment for quality teaching and learning is that we continue to imagine it in terms of early 20th century categories, tools and techniques. We are like accountants trying to run a global economy on the power of the abacus; our objections, fears and even halting efforts to move forward, are all filtered through and inhibited by our imaginations shaped by the framework and capabilities of abacus-skills, techniques and practices.
In this mode, the only way to overcome what we see as a worse possibility of imposed, standardized testing-as-assessment, is to object to what we fear and resist what we can barely imagine. The only assessment we embrace is what we can practice on the micro-level of our classrooms, in the process at the institutional level continuing to use arcane systems of credit-hours, grades, and GPA’s as the feeble currency within and between our institutions; even with this, we find ourselves in the seemingly endless task of negotiating exchange rates between our respective colleges and universities, basing ultimate transfer and articulation value not on what students actually learn but on what we have intended for them, on paper.
To paraphrase an old quote: it is not that authentic assessment has been tried and found wanting; it has not been tried.
There are emerging models of assessment that value as central the micro-level practices of assessment that good teachers already apply in their work. These models look to such assessment as the key source of the evidence of what students actually learn—evidence that is both qualitative and quantitative, that provides a simultaneous source of understanding for both micro-level and institutional-level understanding of the learning that is actually occurring in our institutions.
Fear of assessment, awareness of the worst models of assessment, blind us to the models that could effectively help us to answer our questions (not just someone else’s questions) about what students are actually learning in our colleges and universities.
{I am a full-time faculty, working also as a coordinator of faculty development and student learning outcomes assessment.)
Brian Donohue-Lynch, Dr. at Quinebaug Valley Community College, at 5:25 am EST on November 21, 2007
Gradgrind loves assessment and serves business
Fundamental assumption difference here: either higher education is for the marketplace, a service to employers, as one poster has basically put it, or higher education is for the expansion, refinement, and critical enfranchisement of our society’s citizens.
If the degrees are for sale as commodities, and higher education is some sort of publically funded (subsidized) training arm for American businesses, then let’s just get down to it and do vocational training right out of the gate.
We’re pretty good at that—we know how to create competent lawyers, doctors, electricians, nurses, computer programmers, beauticians, truck drivers, and accountants.
On this view, I don’t see that there’s much value to the traditional B.A. degree. Students should test when finishing high school, pick a career (with the help of a guidance counselor), and the professional and technical schools should add a year or two of remediation, if needed, to ensure that their graduates can communicate clearly and do the minimum required math for the given area of expertise.
In fact, we should seriously consider following the Swiss model of business apprenticeships: For example, not many Swiss bankers have higher education degrees, yet I am told that they are quite proficient at what they do.
Of course, elite institutions will continue to instruct their students in the way they always have, their degrees will still mean what they have always meant, and they will not be beholden to business when their faculties devise courses of study for their graduates. And more than likely their students will, on the whole, continue to lead the world (and the rest of our nation) in educational and career achievement.
Dr. Swifty, Dr. Swifty at Small college, at 5:25 am EST on November 21, 2007
Advertisement
Jobs Related to Another Call for Assessment
or search for jobs directly.
Associate Director, Bridge Year Program
Princeton University
Position Summary: Princeton University seeks an associate director to partner with the director to develop ... see job
Temp — Fire Protection Specialist
Cornell University
Located in Ithaca, N.Y., Cornell University is a bold, innovative, and inclusive teaching and research university of ... see job
Senior Director, Academic Computing
University of Minnesota, Twin Cities
The University of Minnesota is a premier employer and a talent magnet attracting leading faculty and staff from around the ... see job
Director of Major Gifts, Western Region
University of Pennsylvania
The nation’s first university, Penn is a world-renowned leader in education, research, and innovation. Situated on a ... see job
Associate Director — Leadership Coach
University of Minnesota, Twin Cities
The University of Minnesota is a premier employer and a talent magnet attracting leading faculty and staff from around the ... see job
PT Director Center for Innovation Management
Fashion Institute of Technology
FIT Where Creativity Gets Down to Business see job
Media Director
University of Pennsylvania
The nation’s first university, Penn is a world-renowned leader in education, research, and innovation. Situated on a ... see job
Director of Lps International Students & Elp
University of Pennsylvania
The nation’s first university, Penn is a world-renowned leader in education, research, and innovation. Situated on a ... see job
Assoc Dean and Exec Dir International Programs
University of Pennsylvania
The nation’s first university, Penn is a world-renowned leader in education, research, and innovation. Situated on a ... see job
Administrator II
Cornell University
Located in Ithaca, N.Y., Cornell University is a bold, innovative, and inclusive teaching and research university of academic ... see job
Home
| Sign up | Log in
| About us
| Work for Us
| Press Releases
| Rights and permissions
| Contact us
| Advertise
| Partners
| RSS 
| Podcasts
| Links
| Privacy
| My Account
© Copyright 2008 Inside Higher Ed
Getting Ahead of the Curve
This seems like an interesting attempt to get ahead of the curve as far as assessment is concerned and the groups involved should be applauded.
Outcomes measures in higher education are important. Taxpayers are paying billions of dollars to subsidize higher education and the sector needs to be able to show some results.
The problem, of course, is that outcomes measures are context-sensitive. They depend on financial and human inputs. And they depend (or at least they should) on institutional mission.
Some people use this as an argument against more outcomes measurement. But this is wrong: it’s actually an argument against using a *single* outcome measurement, unadjusted for contextual factors.
If HE groups get together and lead the debate about outcomes assessment, the sector can ensure that outcomes assessments are done in a contextually sensitive way. If they squabble about measurement — or worse, deny that outcomes measurements are a good idea — then we may be stuck with a simple-minded, context-free system imposed by governments. And that would be bad for everybody.
Alex Usher, Vice-President at Educational Policy Institute, at 6:05 am EST on November 20, 2007