Feb. 15, 2006

No College Left Behind?

The nerve wracking parlor game of choice for many people in higher education these days is trying to predict where the Secretary of Education’s Commission on the Future of Higher Education is heading. But one thing has become clear: The panel, or at least its chairman, Charles Miller, believes that colleges must better measure the skills and knowledge they impart to students, and openly share that information with the public.

In its simplest form, Miller is advocating “testing” of what students learn while in college. Details — on what measures to use, how to present the information and, perhaps most importantly, whether the testing would be encouraged or mandated — are few at this point, though Miller pointed the way in a memo he sent last month to commission members and in some of his public comments.

Looking for a job?

See all 208 new postings

Browse all job listings:

Faculty: 3,375
Administrative: 1,315
Executive: 170
FEATURED EMPLOYERS

The bottom line: He believes that effective tools for measuring student learning now exist, and that instituting an accountability system that measures and reports student learning is essential, for higher education and for society. “We need to assure that the American public understand through access to sufficient information, particularly in the area of student learning, what they are getting for their investment in a college education,” Miller wrote in his memo.

What is less clear — and this, ultimately, is the $64,000 question — is whether such testing and reporting would happen at the national (or federal government) level. In other words, might the commission propose that all colleges use the same test, or set of tests, to measure their students’ performance, in a way that would let consumers and policy makers make direct comparisons among institutions? On that question the commission, and Miller, have been relatively silent, although when pressed at a meeting this month, the chairman said that he doesn’t “see any way to regulate” or “mandate” the collection and reporting of such information.

For most higher education officials, reaction to the commission’s possible proposal on testing rises or falls on the answer to that question. Many of them agree that emerging tools have made it possible to measure student learning in some critical areas and to begin to assess the “added value” that individual colleges pass on to their students.

And a growing number of college leaders (though not all) also agree that higher education institutions, individually and collectively, must do a better job proving to the public that they are successfully educating students — partly because the current political and economic climate demands it, and partly because it’s the right thing to do.

“Higher education cannot drag its feet,” says Lee Shulman, president of the Carnegie Foundation for the Advancement of Teaching. “It is time for us to do comprehensive, multifaceted assessments [of what students learn on campuses], and make that data public.”

Support for the idea tends to fall apart, however, at the notion of creating a national — or certainly a federal — standard that would apply similarly to all colleges, and that, in the worst case, might eventually be used as a basis for rating or even rewarding or punishing colleges. Here, the specter of the Bush administration’s No Child Left Behind program in elementary and secondary education looms large: College officials fear an overly simplified, one-size-fits-all approach that can’t possibly capture the differences in the missions and student bodies of major research universities and community colleges, liberal arts colleges filled with 18- to 22-year-olds and adult-focused for-profit institutions.

“Trying to create an über-instrument where we simply draw the line and say, ‘This is the measurement,’ will be a grave disservice to the individuals, the institutions and the country,” says David L. Warren, president of the National Association of Independent Colleges and Universities, which represents 1,000 private colleges (where opposition to the testing idea is strongest, as private institutions are less accustomed than public ones to such scrutiny). “We will get a meaningless outcome at a great cost.”

Avoiding that and finding some kind of middle ground — identifying a meaningful way of reporting student achievement while avoiding the trap of an oversimplified national standard — may be the single biggest puzzle facing the commission.

Accountability Pressure Grows

The push for higher education accountability is hardly new. State legislators and members of Congress, accrediting groups, and others have for years been pushing and prodding colleges to justify the mammoth influx of public funds by proving, in measurable ways, that they are successfully fulfilling their many, varied missions. Although many individual state college systems, accreditors and institutions have crafted sets of data aimed at gauging various aspects of institutional success, academics have largely rebuffed calls to measure learning in a systemic way.

“Higher education has deflected the idea for the past quarter century by arguing that the kinds of things we want undergraduate education to teach are not really measurable,” says Patrick Callan, president of the National Center for Public Policy and Higher Education. “There’s been this idea that we’ll just pull some standardized test off the shelf, resulting in a dumbing down of what higher education means.”

The situation is changing in two ways. First, the pressure on higher education to prove itself is mounting, driven most significantly by perceptions that America’s economic competitiveness is slipping as other countries invest more heavily in higher education. Adding to the scrutiny was last fall’s release of a federal study that found only a quarter of American college graduates to be “proficient“ on a set of literacy measures. The results were seen as evidence by observers — including Miller, chairman of the federal higher education commission — that colleges may not be serving their students well, and that the only way of knowing for sure would be to measure student learning more directly.

The other significant change in the climate is that years of research into assessment have, by most accounts, greatly improved the tools available to measure what students learn. From the National Survey of Student Engagement to a slew of institutionally developed exams to the Collegiate Learning Assessment — which is emerging as a favored test in several state and national efforts to measure student learning — “the assessment business has become hugely more sophisticated,” says Callan.

“It has now been demonstrated that it is possible to measure what students learn, and we can no longer rest our case on the argument that it’s impossible,” he adds.

Miller seems especially enamored of the Collegiate Learning Assessment (CLA), which was developed by the Rand Corporation and is now administered by the Council for Aid to Education. The exam has been largely flying beneath the radar until recently, as its makers have kept a relatively low profile as they seek to build what they consider an airtight case about its effectiveness. But the CLA’s supporters have been promoting it aggressively in recent months, and states such as Texas and groups of private institutions have incorporated it into their initiatives to assess student outcomes.

The test aims to measure students’ critical thinking, analytic reasoning, and written communication skills through a series of “performance tasks” and “writing prompts.” In one sample question, students are presented with newspaper articles about the crash of a private plane, federal reports about the accident, charts and other information, and asked to write a memorandum for a company contemplating buying the kind of plane involved in the accident. Test takers are also assessed on how well they can support or critique a stated point of view.

While its sponsors eventually would like campuses to use the CLA longitudinally — measuring the same group of students as they enter as freshmen and when they leave as seniors — institutions now typically give the test to a sample of 100 freshmen and 100 seniors.

Richard H. Hersh, the former president of Trinity and Hobart and William Smith Colleges who co-directs the CLA project, says he and others responsible for the test believe they are producing “valid and reliable data now” that show it is possible to measure “real learning gain as a function not only of the fact that you’re in college, but where you attend.”

Fans of the test are already sold. “We believe that the CLA provides a robust and flexible tool that allows higher education institutions of a very wide range of characteristics to assess specific kinds of cognitive outcomes among students – the kinds of macrolevel changes in students that you hope will happen over course of a four-year education,” says Geri H. Malandra, associate vice chancellor for institutional planning and accountability at the University of Texas System, which has incorporated the CLA into its system for assessing the performance of its nine institutions on a slew of measures. (Miller, the chairman of the federal higher education commission, helped to get the system in place when he headed the Texas Board of Regents.)

But Malandra acknowledges that “there isn’t any approach to learning assessment that would be sufficient on its own,” which is why the Texas system incorporates students’ scores on the National Survey of Student Engagement and on state certification exams, among other factors, into its assessment of what students learn at its institutions. “The literature on assessing student outcomes is very clear on this — multiple frames [of measurement] is what you need.”

Or, in the words of Carol Geary Schneider of the American Association of Colleges and Universities, who sits on the Council for Aid to Education’s board: “I like the CLA, I think it’s a breakthrough, but it is by no means the solution.”

A ‘Marriage of Insufficiencies’?

That last point is crucial, even to the growing core of people in and around higher education who agree with Miller that it is possible, and politically necessary, for colleges and universities to better measure their success in educating students.

Robert J. Sternberg, the renowned psychologist and new dean of arts and sciences at Tufts University, largely agrees with Miller that parents, students and lawmakers are “paying a lot of money [for higher education] and they ought to know what they’re getting.” But an accountability system done badly could be worse than no system at all, Sternberg says. The CLA is “good as far as it goes,” he says, but it is far too narrow a measure of what students should learn in college. It ignores crucial skills such as creative thinking and the ability to collaborate with others – skills that Sternberg hopes to capture in a project he is beginning at Tufts that defines “ ’value added’ broadly.”

As president of the Teagle Foundation, which is sponsoring a series of grants in “value added assessment,” W. Robert Connor, too, calls it “an impossible position” for faculty members and college administrators to “say that we don’t want more knowledge about our students and what they’re learning.” But Connor also agrees with Sternberg that despite “really good progress” on the CLA and other assessment tools, “there’s a lot that’s still to be done” in developing an effective system for measuring student learning.

Connor recognizes, he says, that higher education leaders “can’t sit there and say, ‘We’ll have better instruments in 10 years, so let’s wait until then’ “ to put in place an accountability system. But it would similarly be a mistake for the federal commission, or anyone else, to impose a top-down, flawed solution on academe, which is why, he argues, that “the way to deal with this is for higher education to get out in front and do it right.”

College leaders, urged on by the commission from its bully pulpit, “can do a much better job of assessing students’ progress in the development of important cognitive skills, using those measures that are available now while at the same time pushing ahead on finding other, better measures,” Connor says. Shulman of the Carnegie Foundation agrees, saying that “there ought to be an expectation that every institution takes on responsibility for demonstrating the ‘value added’ of their educational programs for their students, but recognizing that institutions vary so enormously in who their students are, and what their missions are.”

Advocates of this approach point to efforts like the one adopted by the State Council for Higher Education in Virginia, which in 1999 began requiring public institutions there to gauge and report their own performance in a range of areas, including student learning, but left it to the individual institutions to decide which measuring sticks to use. Or, Shulman says, groups of institutions might collaborate to identify what he calls a “marriage of insufficiencies” — a “carefully designed suite of assessments,” each of which might be “deeply flawed,” but “collectively is a robust and most sensitive set of measures.”

Foot dragging by colleges will not do, Shulman says. “The challenge is for higher education institutions to make some proposals that would guarantee that within a year, you begin to get policy-useful data generated by some of these approaches.”

If Shulman, Connor and others represent higher education’s attempt to find what Schneider of the American Association of Colleges and Universities, calls a “middle ground” on the testing issue, Stanley N. Katz reflects the reality that many people in higher education would prefer a more combative approach.

“If you think that No Child Left Behind is good for the schools, you’re likely to think this is good for the colleges,” says Katz, a professor at Princeton University and former president of the American Council of Learned Societies. Any attempt to define across the great variety of higher education institutions a common set of standards or measurements of what students should learn, Katz says, will be doomed: “Either there won’t be agreement, and it will be overly controversial, or it will be reduced to an elastic, lowest common denominator, as in No Child Left Behind, in which case it will become trivial.”

Katz insists that he is not saying colleges do not need to be accountable — he just thinks they already are, by students and their families who are continuing to pour into the institutions. “There will always be legislators and legislatures that would like more bank for the buck, bigger results for less money,” he says. “But I think the public is quite satisfied with what higher education is doing on the whole. This is a market system, and the customers are buying. We have by a considerable measure the finest system of higher education in the world. And if that’s the case, this is an ‘ain’t broke, don’t fix it’ situation.”

He adds: “And I think some well-placed university presidents ought to pull up their socks and say that.”

If college leaders agree with Katz, few if any have been willing to say so publicly thus far, partly to avoid prejudging the work of the federal commission and partly out of fear of looking like knee-jerk obstructionists. But over the next few months, as the panel crafts its recommendations that are due to Education Secretary Margaret Spellings August 1, one set of higher education officials may find themselves in an especially well-placed (but perhaps unenviable) position: those current and former college presidents who are members of the federal commission.

To date, panel members like David Ward, president of the American Council on Education, and Charles M. Vest and James Duderstadt, former presidents of the Massachusetts Institute of Technology and University of Michigan, respectively, have largely refrained from any overt criticism of the testing concept, preferring gentle suggestions that the panel favor exhortation and agenda setting over top-down mandates. And Miller, the commission’s genial chairman, has so far said the things they want to hear in response.

But should the panel begin to shift its focus toward a more “consumer friendly” approach that would allow easy comparisons across institutions — which would only be possible through a commonly applied set of standards and measurements — the college leaders on the panel will be in the hot seat.

— Doug Lederman

Want it on paper? Print this page.
Know someone who’d be interested? Forward this story.
Want to stay informed? Sign up for free daily news e-mail.

Comments

Inevitable

Given the results of recent literacy and numeracy surveys, tuition and fees spiraling out of control, and corresponding demands for government subsidies at virtually every school in the nation, exit testing and accountability are inevitable. Further, they should be welcome: There is nothing unreasonable about assuring that colleges and universities produce literate and numerate students. If only faculty and administrators had taken their professional responsibilities seriously for the past few decades, and had not allowed college students to graduate without demonstrating basic literacy and numeracy, such external quality checks could have been avoided.

Bad English, at 9:20 am EST on February 15, 2006

Specious educationism strikes again

But who will assess the assessors? And the assessors of the assessors?

Some of those who want to whip that crowd of evil professors (most of them Democrats!)into submission are also trying to turn universities into imitations of the University of Phoenix, but it’s clear that online courses invite massive fraud and thus undermine the very idea of academic credit.

The academy knows what academic excellence is. But the academy has never faced such competition before from so many media, so many gadgets, so many young people looking for short-cuts. Our job is to get students to study the material we put before them, and that’s a different task than it was even a decade ago. We must resist the political opportunism, envy, and malice of those who are themselves promoting bogus short-cuts to our objective.

Turn the university into a realm of bean-counters and goose-steppers, and I’ll guarantee your children a heck of an education, Brownie.

Hnaef, at 10:15 am EST on February 15, 2006

Time to come clean

The industry’s assessment challenges and opportunities have been discussed ad nauseam. Professional organizations and legislative panels have focused on the subject for years with little or no progress. In the abstract assessment is an unassailable ideal. In reality, we consistently affirm assessment is a slippery concept that must be approached with care. The sanctity of academic freedom is at stake.

What happens in the classroom, stays in the classroom, only a grade on a transcript or an entry in a portfolio scream out. The solution is both simple and threading, assess with the individual student knows and can do at entry and again an exit. Value added will be manifest.

The route to value added will not be traversed by more hearings and studies but by adoption by individual faculty members willing to take risks for the benefit of their students. Those who success will validate what they have known in their hearts. Those failing will have a challenge to improve their service to their students

Pat Leonard

Pat Leonard, Vice President for Academic Services at College of the Southwest, at 11:00 am EST on February 15, 2006

The Source of Discontent

It is important to remember that one source of dissatisfaction with higher education in the U.S. is faculty opinion. After 25-years of teaching and serving as an administrator, it is my observation that most faculty believe they are doing a good job but most everyone else has succumbed to external pressures and is contributing to grade inflation. Although, I think the CLA (or other standardized tests) have some value for cross institutional comparisons, the solution I advocate is the institution of faculty developed comprehensive exams which are administered to graduating seniors in capstone courses. General education objectives could be assessed by requiring students to take a qualifying exam before declaring a major. The results of the qualifying exam could be used (along with GPA and other factors) in determining who is admitted to competitive programs. Assessment would be controlled by faculty and such a scheme has an academic precedent in the sequence typically required for a doctoral degree

Michael Poteat, Director of Institutional Effectiveness, at 11:25 am EST on February 15, 2006

This is not the university it was

Much of the concern within The Academy disappears when one realizes that the University that was is not the university of today.- the concern over testing becomes a different question when one understands that school is really K-16 and faculty in grades 13-16 are expected to provide the same as teachers in K-12 or at least 10-12.

- faculty, too, are in new roles and in many disciplines, particularly outside the natural sciences, are being questions, not just from the outside (conservative vs liberal) but from within their own disciplines; economics, political science and literature or the humanities in general are good examples

-both faculty and now student qualifications are increasingly being questioned both from within and outside the institutions as are the instutions themselves as commodification and market forces have breached the walls of the ivory tower.

In order to defend The University, if that is what one must, should or could do, then one must be clear what one is defending.

tom abeles, editor at on the horizon, at 12:30 pm EST on February 15, 2006

It was about time.

Here we go again with the hot potato in the history of contemporary higher education: assessment of learning. Simply put, student learning outcomes assessment is a very direct way to measure the quality of learning (or the lack of it). The thing is how we go about measuring something so complex and multilayered as learning. Simply answer, by doing it! Whenever there’s an excuse to gauge the quality of something, it’s because we are not sure of how good the product is and we just do not want to know. I wonder if professional schools such as medical schools have got the same problem when deciding to measure whether their residents know how to perform an appendectomy or not. I bet they came up with standards and criteria that guarantee effective performance from every student who holds a diploma in surgery. The same should apply to students holding a diploma in Foreign Language education, arts, history, maths, etc. If the professional organizations pull together for the implementation of performance-based assessments, I am sure that some common ground will be reached. Higher education has a lot to learn from professional and trade schools.

José Ricardo, Assist. Professor of FL Education and Spanish at Shippensburg University, at 12:30 pm EST on February 15, 2006

We have given away our power, and now we are shocked that others have taken it. We have allowed those without academic understanding to impose their will in our classrooms, and now we teach what they want instead of what we know to be necessary and true. We have only ourselves to blame. Shame on us!

Ann L, Professor, at 12:30 pm EST on February 15, 2006

Big Brother

The idea of the federal government dictating learning outcomes through assessment frightens me. This development threatens to align curriculum with tests, which provides politicians and their funding agents with inroads into the classroom. High stakes testing in the academy will redefine collegiate learning from a complex mesh of leadership, spiritual, intellectual, social, moral, and identity development to a rudimentary knowledge transfer process.

David Franklin Ayers, Assistant Professor at UNC Greensboro, at 1:35 pm EST on February 15, 2006

Here’s a question for all you liberal arts Ph.D folks. It comes directly from my son’s seventh grade math workbook:

A group of kids buy sodas and candy bars. They spend $3.36 on soda and $2.45 on candy. Each kid has one can of soda and one candy bar. How many kids were in the group? How much does a soda cost? A candy bar?

Can’t figure it out? Does this mean you shouldn’t have even graduated from junior high? Are you going to turn in your Ph.D.?

I’m shocked, absolutely shocked, at your lack of numeracy.

Philip, English teacher, at 2:05 pm EST on February 15, 2006

So That’s Why They Must Be Tested

Philip’s question and his profession interest me. I recall telling my Algebra I student years ago that if you can read, you can solve these problems.

Now for the real reason those legislators are so hot for testing these days ...

Math in 1950

A logger sells a truckload of lumber for $100. His cost of production is 4/5 of the selling price. What is his profit?

Teaching Math in 1960

A logger sells a truckload of lumber for $100. His cost of production is 4/5 of the selling price (i.e., $80). What is his profit?

Teaching Math in 1970 (Constructivist Math)

A logger exchanges a set “L” of lumber for a set “M” of money. The cardinality of set “M” is 100. Each element is worth one dollar. Make 100 dots representing the elements of the set “M". The set “C", the cost of production, contains 20 fewer points than set “M.” Represent the set “C” as a subset of set “M” and answer the following question — What is the cardinality of the set “P” for profits?

Teaching Math in 1980

A logger sells a truckload of lumber for $100. Her cost of production is $80 and her profit is $20. Your assignment — Underline the number 20.

Teaching Math in 1990:

By cutting down beautiful forest trees, the logger makes $20. What do you think of this way of making a living? Topic for class participation after answering the question — How did the forest birds and squirrels feel as the logger cut down the trees? There are no wrong answers.

Teaching Math in 1996

By laying off 40% of its loggers, a company improves its stock price from $80 a share to $100 a share. Assume that capital gains are no longer taxed ... because this encourages investment. How much capital gain per share does the CEO make by exercising his stock options at $100?

Teaching Math in 1997

A company out-sources all of its loggers. The firm saves on benefits, and when demand for its product is down, the logging work force can easily be cut back. The average logger employed by the company earned $50,000, had three weeks vacation, a nice retirement plan and medical insurance. The contracted logger charges $50 an hour. Was outsourcing a good move?

Teaching Math in 1998

A laid-off logger with four kids at home and a ridiculous alimony payment from his first failed marriage comes into the logging-company corporate offices and goes postal, mowing down 16 executives and a couple of secretaries ... and gets lucky when he nails a politician who is on the premises collecting his kickback. Was outsourcing the loggers a good move for the company?

Teaching Math in 1999

A laid-off logger serving time in Folsom Prison for blowing away several people is being trained as a COBOL programmer in order to work on Y2K projects. What is the probability that his automatic cell door will open by itself as of 00:01, 01/01/00?

Teaching Math in 2000

A logger sells a truckload of lumber for $100. His cost of production is $120.How can Arthur Andersen determine that his profit margin is $60?

Teaching Math in 2010

El hachero vende un camion carga por 100 pesos.La cuesta de production es ...

RWH, at 4:25 pm EST on February 15, 2006

Gird for Battle

Fed up with underperforming and exorbitantly costly campuses, the public is rightfully demanding evidence of students’ literacy, numeracy and critical thinking skills. It is long overdue to institute, as Commission Chair Miller advocates, a higher education accountability system focused on student learning. And, it is entirely feasible to reliably measure, clearly report on, and validly compare on a national campus-by-campus basis real learning gains – gains geared to standards defined by distinguished academics themselves.

But if my and others’ years-long struggle to instate such a system at the State University of New York is any indication, the Secretary of Education and her colleagues can expect ferocious resistance to, specifically, comparisons of institutions’ performance and transparent reporting of assessment results. In the end, the SUNY leadership bowed to denunciations of “invidious” and “un-collegial” comparisons which, as expressed in official assessment guidelines, “would tend to diminish other institutions,” and it agreed to the “public dissemination of assessment data…only through aggregate reporting” (thus effectively concealing individual campuses’ poor performance) as well as to the sharing of such data “only with appropriate stakeholders” (that is, to the exclusion of the public).

Getting higher education to let us in on how well it is serving students and the nation will, in short, be a long and arduous battle.

Candace de Russy, Trustee at State University of New York, at 4:25 pm EST on February 15, 2006

What’s your point?

7 kids, 48 cents, 35 cents.

RW, at 4:25 pm EST on February 15, 2006

Diversity of opportunities makes us strong

Doug Lederman is a solid higher education reporter who should know better than to infer that independent colleges are opposed to national testing because “private institutions are less accustomed than public ones to such scrutiny.” While independent colleges are not subject to direct state government control, the accountability for both public and private colleges is virtually identical in all other respects. If private colleges do have a heightened sensitivity to poorly-thought-out accountability measures like a national test for college graduates, it is because our institutions, the programs they offer, and the students they serve are very diverse. Using a single yardstick to measure institutions’ success in producing artists, accountants, engineers, actors, social workers, and biologists oversimplifies what our institutions do, and what our students accomplish in their years with us. Independent colleges are not opposed to assessment. We are opposed to a one-size-fits-all federally imposed test that would kill the diversity of opportunities in higher education – the very quality that makes us strong.

Violet Boyer, President at Independent Colleges of Washington, at 4:25 pm EST on February 15, 2006

elephant in the room

One thing that hasn’t been mentioned is the fact that at many universities—professors are not hired and evaluated (ie tenure) based on their ability to TEACH. And if a student is at a big university in classes with a few hundred other students, how are they learning critical thinking and writing skills? By taking multiple choice tests? Going to a TA section once a week?

Are we going to change the whole structure of employment and tenure to actually (and acurately) reflect the importance of student learning? When you are on a tenure committee at a university that has 20,000 undergrads, the main discussion isn’t about studenet learning but what kinds of grants and prestige a potential hire is going to bring to your department.

While there are many small colleges, community colleges that focus on student learning, the reality is that for a majority of college students they are NOT the priority of the college or university.

Sustaining the existence of the collge or university IS THE PRIORITY(i.e.bring on the NSA grants, tenure is based on the # of publications are in the top journals, or sustaining a winning sports program to bring in the money as well).

So lets start looking into the mirror here before we judge...

com college prof, at 4:25 pm EST on February 15, 2006

Program Assessment

If we are to assess programs they need to be standardized. In my discipline (Computer Science) this is not the case. CS programs tend to be all over the map.

We’ve looked at the MFAT for CS and it simply does not fit our program.

In addition I wonder about the credentials of those writing these tests. In our case the exams are not coming from the professional organizations (ACM, IEEE) but rather a testing service.

Our assessments (which we do need to have) will probably be based first on placement data: is industry hiring our graduates?are our graduates entering graduate programs? and feedback from our internship programs.

Some other issues:

In a field where change is the norm we’d have to freeze ourselves for an exam.

What people want to know about our students is whether or not they can build or modify systems. That is not something we evaluate by a standardized test but rather by actual performance on projects (a portfolio of successful work).

Rob Rittenhouse, Assoc. Prof at McMurry University, at 4:25 pm EST on February 15, 2006

Seeing & Educating Like a State

For a wonderfull essay expanding James Scott’s thesis (from Seeing Like a State) to examine trends in educational standardization see: David Price’s critical, “Outcome-Based Tyranny: Teaching Compliance While Testing Like A State” Anthropological Quarterly (2003) 76(4):715-730. On page 719 Price writes: “Today a lucrative industry of test designers (estimated to be worth between $700 million and one billion dollars ayear) is followed by a kowtowing curriculum industry rolling across America like a fleet of Ambulance chasers—pitching textbooks, work sheets and bric-a-brac designed to help districts more effectively “teach the tests.”

Follow the money.

Fester L. Quanticus, at 4:40 pm EST on February 15, 2006

Oh Wow!

Hey look you guys, this idea of holding colleges and universities accountable for the quality of their “products” is quite wonderful.

I have already filed papers of incorporation – in all fifty states – for two companies. The first is a College Board-like company that will provide testing programs for Federal and State Governments ... and the second is a Kaplan-like company that will advise and tutor the colleges and universities and help them learn whatever is required to pass the Fed’s tests.

Needle$$ to $ay, thi$ ha$ got to be the greate$t idea $ince $liced bread.

Bring it on!

RWH, at 5:05 pm EST on February 15, 2006

Cheaters, sluggards, and chowderheads

I solved the math problem involving soda and candy—but I have only an M.A. in English and haven’t suffered the brain damage caused by a humanities Ph.D. program.

Unless I’ve missed something, no one has commented on a couple of obvious things. First, to test “outcomes” or “student learning” will only encourage—indeed, demand—students, teachers, and administrators to cheat. Human nature always wants to beat the system. Sure, a person should have to demonstrate knowledge and skills before getting any sort of credential, and passing a test (feel free to substitute whatever euphemism you prefer) is how one demonstrates that one has learned. When tests have bad consequences—not getting a diploma or degree, losing a job, losing funding, or whatever—people worry first and last about avoiding the consequences. This problem has no solution. Well, it has the same solution as a lot of other problems: we need to be better people than we are. That has always amounted to the same thing as “no solution.”

Second, the endless blithering about assessments, outcomes, student learning, transparency, and accountability rarely contains certain words. In discussions of student learning in higher education, one hardly ever hears or reads such terms as effort, will, diligence, character, or intelligence used in reference to the students. But laziness exists. Stupidity ditto. There’s no dodging an ugly truth: Not everyone can learn everything. Hell, most of us can’t learn much at all.

So let’s have several levels of tests—challenging, nationwide exams prepared by mean old schoolmarms. We can create a Bureau of Mean Old Schoolmarms to make and grade the tests. Let’s have the tests change from year to year, and let’s keep them secret until they’re actually handed to students. A person who doesn’t pass the United States High School Entrance Exam doesn’t get out of middle school, even if he’s 17 years old. Someone who fails the U.S. High School Exit Exam doesn’t get a diploma, even if he was captain of the football team. The U.S. College Level Entrance Exam will determine who can apply for admission to college. A person who doesn’t pass the U.S. Bachelor’s Degree Exam doesn’t get a degree, even if she has spent loads of money racking up credits.

How will students know what to study and teachers know what to teach? They won’t. They’ll simply need to learn and teach as much as possible, with particular emphasis on basic stuff such as reading, writing, reasoning, doing math, and knowing important facts, formulas, and methods. The tests will be fair; but they will not let everyone pass.

I’m serious. And let’s give the whole program a truthful, attention-getting name: Your Child Cheerfully Left Behind. That’s honest. Realistic, too, since the real world will cheerfully leave any individual behind and then come back to strip the corpse. Under YCCLB, people who want credentials will have to KNOW THINGS and prove that they know. Those who can’t or won’t learn won’t get credentials.

High-school enrollment will fall. High-school graduation rates will sink. College enrollment will plummet. College graduation figures will shrivel.

But would the nation really know less under YCCLB than it knows now? Or would we merely have fewer people with certificates of knowledge?

Baktu Basix, Weary teacher, at 7:10 pm EST on February 15, 2006

Two More Problems with Testing

I agree with many of the criticisms levied against using standardized tests as a means of determining how well colleges and universities teach what they purport to teach, but there are two points that have not yet been raised.

First, unlike K-12, the students at universities are adults, and they are responsible for their decisions. Consequently, they are at least as responsible for learning as the professors. To be blunt, are the professors at Yale who attempted to teach George W. Bush the fundamentals of English grammar responsible for the fact that their student evidently learned nothing?

Second, at my institution, at least, it is far from unusual for students to transfer in after spending a number of years elsewhere, sometimes at community colleges, sometimes at other universities. If a transfer student does badly (and again, a very significant proportion, possibly a majority, of SDSU’s students are in this category), is it SDSU’s fault? Are we to be held responsible if another institution does its job poorly?

Finally, it is very curious to me that the proponents of testing are not interested in letting the free market do its work for them. If a college or university stops answering a need, then students will likely stop going there, just as a bad product will not find buyers. Students flock to American’s institutions of higher learning. There is considerable worry at my institution over our ability to handle the expected crowds. If we did not provide a product that the public desires, would our enrollments, and applications, continue to rise?

Peter C. Herman, Prof. at SDSU, at 8:50 pm EST on February 15, 2006

“It’s always Bush’s fault”

” .. To be blunt, are the professors at Yale who attempted to teach George W. Bush ..”

Why, yes .. and that statement is about as relevant as the statement, “all public college professors are biased Communist traitors and should be fired.”

That statement plays right into the current David Letterman running gag “if there’s something wrong — it’s Bush’s fault.” Grow up, will ya?

Bart J., at 6:15 am EST on February 16, 2006

the K-12ization of Higher Ed..

This means that america will fall behind the rest of the world in higher ed, too. We can just leave our poor status in K-12 ed, alone!!

Sonya, at 4:55 pm EST on February 16, 2006

Too many differences

Every college and university has a different mission, vision, and goals that directs how that institution will educate.

As stated already, how then do we create a “test” to measure what should be learned when what should be learned is really up to the institution’s priorities, mission, vision, and values.

What this proposal would lead to is a national curriculum for higher education... and that is an intrusion that would kill the spirit of higher education. What a devastating blow to academe.

Peter, at 4:55 pm EST on February 16, 2006

Handwriting on the Wall?

Doug Lederman’s “No College Left Behind?” and related comments provide an excellent summary of the issues surrounding outcomes assessment in higher education [InsideHigherEd.com, Feb. 15, 2006].

How can one not agree with Charles Miller, Chairman of the Secretary of Education’s Commission on the Future of Higher Education that colleges must better measure the skills and knowledge they impart to students, and openly share that information with the public? The government ought to be the first to have this information. Why? Because the feds are now subsidizing higher education and the NCAA with favorable tax policies and no end of financial support with little or no feedback on the return on the American taxpayers investment.

From a taxpayers point of view, a good place to start assessment would be with the athletes from colleges and universities that maintain big-time (Div 1A) football and basketball programs — limiting comprehensive, multifaceted assessments to the top one-third of the players having the most playing time on each team. This would provide aggregated data from a cohort of about 35 athletes from each school — 30 from football and 5 from basketball.

Candace de Russy is quite correct in saying that getting institutions of higher education to make public information on how well they are serving students and the nation will be a long and arduous battle. As Paul Gallico wrote (with reference to the Amateur Athletic Union) some 70 years ago in FAREWELL TO SPORT : “One of the easiest things in the world is not to have evidence when evidence is liable to prove embarrassing.”

I also agree with Lee Schulman that institutions of higher education cannot drag their feet — putting off the implementation of a “carefully designed suite of assessments,” each of which might be “deeply flawed,” but “collectively is a robust and most sensitive set of measures.” One element of the suite should be a list of the courses taken by each cohort, should be the average grades for all students in those courses, and the names of advisors and professors who teach those courses.

All of this amounts to handwriting on the wall for the NCAA. As Supreme Court Justice Louis Brandeis stated: “Sunlight is said to be the best of disinfectants.”

Frank G. Splitt, Member at The Drake Group, at 4:55 pm EST on February 16, 2006

While there may be many arguments as to the efficacy of any tests or monitoring systems of universities, who wishes to ‘run’ these tests remains an most important question. Already, the Federal gov’t is finding ways to circumvent the idea that education is within the State’s venue with NCLB. With this, the agenda and pocketbooks of many politicians have managed to creep into the classroom. We would be foolish to assume that political figure WON’T make politically motivated decisions when it comes to higher education as well, especially if we give them a tremendous amount of power over them.

National government should not be making decisions about what the colleges are teaching. While the universities often don’t go a good job of self-evaluating, based on the last decade, we certainly can’t count on the government to do any better.

Shauna, Educator, at 6:25 pm EST on February 16, 2006

I appreciate that a previous poster mentioned that professors are often not hired according to their ability to teach. Institutions of higher education sometimes feel that their faculty’s publications and research are more important than their ability to effectively impart knowledge to students. It’s unlikely that the students would have the same priorities as department chairs and administrators, leading to an agency problem. At most institutions, students are the primary source of revenue, and a major reason for the existence of higher education in the first place.

As a graduate student, I’m not terribly interested in professors’ own research. I’m much more interested in their ability to dissect and discuss the major contributions to the field in which they teach.

Perhaps the commission could start with this issue.

Russell Dover, at 7:40 pm EST on February 16, 2006

I’m just a parent of a girl in Berkeley’s ugrad engineering school — not an academic. But doesn’t that make me an informed consumer of higher education’s end-product. I see some tunnel vision here in prior messages.

The Fed’s do not have to mandate the tests to be performed within or by “higher ed.” Many students go through the FAFSA fin. aid. process. The govt. already mandates a counseling step upon entry into indebtedness (i.e. into higher ed.) and upon entry into repayment responsibility (at least sometimes associated with graduation). The tests could be administered by an entirely separate bureacracy or industry — in which case ALL higher ed. academics could be disintermediated from the process. Isn’t it better to be participating in one’s own assessment conducted by outsiders? If academia resists quality assurance too strongly, the assessment will simply be conducted clinically and entirely outside of academia.

Bruce Harvey, at 4:25 am EST on February 17, 2006

fyi: Stallings Commission home page

This may be helpful — straight from the source —

http://www.ed.gov/about/bdscomm/list/hiedfuture/index.html

R.A. Shaw, at 2:50 pm EST on February 18, 2006

Assessment is Alive and Well in Higher Education

Measuring a high quality education is a bit like recognizing pornography: it is hard to define, but you probably know it when you see it. Do we give up the fight against pornography because it is so difficult to define? I certainly hope not. The key to any successful policy implementation lies in the details, but you do not give up simply because it is difficult to implement, as is the case with assessing outcomes in higher education.

Some of the misguided paternalistic comments here suggest that only academia can save students from the perils of consumerism in higher education, but then argue that the complexity of measuring learning puts it beyond the effective reach of regulation. It is a personal challenge to stave off cynicism and simply assume that academic simply refuses to change even as students behave more and more like consumers (i.e. they exercise their ability to inform themselves and then choose) and already hold institutions accountable for outcomes based on their personal experience.

It is my experience in higher education that outliers garner a disproportionate share of attention. Believe it or not, most students do not seek a “cheap” solution or an easy degree. Students know a valuable education when they receive one, and are quick to tell their family, friends, and colleagues when they were well served, and when they were poorly served in higher education. Accountability and assessment is alive and well in higher education, but perhaps most faculty are just too afraid to admit it.

Bela Barner, VP of Strategy at Private Non-Profit Institution, at 11:50 am EST on February 21, 2006

I find it interesting that discussions of problems in education seldom, if ever, address the negative impact that part of FERPA (Family Educational Rights and Privacy Act) have had. Not being able to make students’ grades public has created a number of logistic problems for educators and staff, but more importantly, it has allowed students to be lazy and yet be able to hide behind FERPA so that no one (the public) knows. We now see an extension of this way of thinking in anonymous chat rooms and anonymous posting of comments about people on the Internet.

When I went to college, I found out how well I did on a test by going to my professor’s office and seeing my score or grade on a roster that was posted in the hallway. It didn’t stunt my growth or turn us into invalids.

Whenever I have asked a class how they would feel if they knew that their grades would be published in the local newspaper or on the Internet, they have always replied, “Then I’d have to study.”

If there is such an outcry for public information, modify FERPA so that we can truly provide public information. This is a no-cost step that we could take that would help get us back on the correct academic track.

Donald Ramirez, at 3:15 pm EST on March 1, 2006

Program accreditation

Many are commenting on the need for reform or the need not to reform the process of accreditation Is it not the purpose of the accreditation process to account for your strengths and weaknesses, to be self critical, and to correct the shortcomings of your institution? Since no system is perfect, it seems necessary and reasonable for the agencies responsible for accreditation to assess and reform their own standards. I have 5 health science programs that have programmatic accreditation. We are all held to a high standard which include mandatory graduate testing as well as meeting threshold levels for each domain of learning, graduation rates, job performance, and employment. I have found that the regional agencies do not have this standard for the general education areas. Most faculty members in the general education area even have a difficult time setting measurable goals and benchmarks. So, it makes me wonder if regional accreditation agencies are doing what they claim. Are institutions really living up to the standards that they have on paper? I know our institution is not perfect, but the staff, faculty and administration strive to reach those goals everyday. I am sure that most colleges and universities do the same; however, there is a difference in putting forth effort and actually achieving your set goals. If the latter is not being accomplished, the effort is for not. Do you pass students for a great effort or for earning the grade? If the former is your philosophy, you have made Millers point.

William Croft, at 10:00 am EDT on April 4, 2006

Performance Measurement in Higher Education

It might be more to the point from the students point of view, to see employment outcomes of graduates on a program/institution basis, which would be theoretically possible via tracking unemployment wage records, with results controlled for by relevant economic indicators to account for the overall health of the job market.

Yes, I recognize the enormous contribution that our colleges make to the enhanced quality of life of their graduates by teaching them to appreciate new (and often non-economic) things. But let’s face it most went because they (however dimly) conceived it would land them a good job. To what extent does that occur? I’d love to see the results for certain community college programs. Results will be ugly for some programs in 4 year schools as well. Don’t kids have a right to know and think about this? It might lead some to better after school plans.

As for measuring learning competencies of graduates, we may want to step back and determine first why 30-40% of college students (compare to the growing chorus of 30% for high school students) drop out (even accounting for myriad transfers and epic 6-8 year collegial journeys).

If we ever seriously look I think we will find that the root cause has less to do with teacher competency or even choice of major, but instead a weak career development infrastructure within and across levels of education from primary school through undergraduate.

Apply NCLB to higher education seems likely to result in the same kind of dysfunction, goal displacement, (and the occasional cheating) typical for any application of performance measurement where the goal of learning how to improve outcomes is trumped by “accountability” as a campaign slogan.

Greg Lagana, Workforce Policy Analyst at Carnegie Mellon University, at 5:40 pm EDT on April 12, 2006

With testing and accountability comes federally mandated standards, which is tantamount to a federally mandated curriculum. Are you ready for a National Curriculum? Remember the “Bill of Rights” for higher ed. is a thinly disguised attempt to squelch academic freedom. I really think that it’s another way for neo-cons to influence and control... well, everything.

Harbinger, at 12:05 pm EDT on June 22, 2006

Jobs Related to No College Left Behind?

or search for jobs directly.

Assistant Professor: Psychology/Human Development — Job ID 978
SUNY — Empire State College

Empire State College’s Genesee Valley Center seeks applications for a full-time, tenure-track faculty position in the ... see job

Project Director
University of North Carolina, Chapel Hill

Overview: The Project Director will work with the Director in running the programs of the Environmental Finance Center (EFC). ... see job

Information Technology Professional
University of Minnesota, Twin Cities

The University of Minnesota is a premier employer and a talent magnet attracting leading faculty and staff from around the ... see job

Graduate Nursing Program Director
North Dakota State University

Be a part of the positive energy and momentum at North Dakota State University as we expand our graduate nursing program. see job

Assistant/Associate Professor of Structural Geology
East Carolina University

East Carolina University, a constituent institution of the University of North Carolina, is a doctoral institution with an ... see job

Professor, Phychiatry
University of California, Irvine

School of Medicine Department of Psychiatry and Human Behavior MRI Imaging Program Position: Assistant or Associate Professor ... see job

Assistant Professor
Western Carolina University

Western Carolina University is seeking an assistant professor in international relations and/or comparative politics. Area of ... see job

FT 12 Mo Faculty — 2020A
Saint Louis University

Saint Louis University is a Jesuit Catholic University. Through teaching, research, health care and community service, Saint ... see job

Professor and Head
Mississippi State University

Head of the Department of Counseling and Educational Psychology at Mississippi State University. This is a 12 month ... see job

Assistant Professor in Marriage and Family Therapy
Friends University

Lead, Inspire, Teach.....at Friends University! see job