Defining College Success

Start with the student

Same question, same response, just in another form. In some ways the articulated discussion is better and in some ways worse. Standardized tests (pre and post such as the SAT and GRE) as a measure of learning for everyone? I don’t think so. What will that really tell me? External benchmarks? Possibly as part of a response but that’s not the entire response. Diversity is a strength of the educational enterprise. Accountability is essential but to whom are we accountable? One of the prime groups to whom we are accountable is our students. Personal goals are the key to the response. What are YOUR goals? Did you make progress towards them? Have they changed? Do you believe that the exercise and expense (time, energy, money) was worth it? Begin tracking systems. When a student enters, begin a conversation about what their goals are. Small ones are fine. Big ones, we know can probably change, even the small ones will. It’s all a building process and opening of eyes, thoughts and heart. These are articulated goals are a way to continue to check in with students and frame a conversation of direction and growth. They’re also an early warning system of issues. It’s the intersection of dreams and reality. Perhaps those are the conversations and the measures we should have.

Here’s another suggestion: we have outcomes goals that must be measurable. Should some of the outcomes and discussions that we have with students include: think about how long it took you to do x when you started this course. How long does it take you now? What did you find difficult/challenging in the beginning? How do you feel about it now? Identify something that you learned in this course and share how you will be able to apply it to some aspect of your life. Small measures, even in the face of not doing as well as they hoped. Celebrate successes; they add up, but they don’t just happen. They are managed and we are partners in the exercise.

Dorothy Umans, Montegomery College, a community college in Maryland, at 10:50 am EDT on May 20, 2008

Missed opportunity

A clear alternative to defining and reporting college success in educating students is an institution’s process whereby faculty decide expected student learning outcomes and then evaluate regular student work/activity in terms of these. Given sufficient system support, this permits a college or university to generate data on actual student learning outcomes for any set of students (or individual students) directly out of its ongoing curricular and co-curricular program. Perhaps the just-published book refers to this alternative, but I doubt it, given the fact that the College Board was presented the opportunity in 2002 to participate in the development of a system that supports this process and chose not to do so.

David Shupe, Doctor at eLumen Collaborative, at 11:00 am EDT on May 20, 2008

the illusion

The issue has been argued since the modern western university was founded (Newman, Kant, von Humboldt) Few worried when Harvard was founded in 1636 when a high school diploma was a reach. In the current world, an engineering recruiter once said that when he wanted engineering management he went to east or west coasts and when he wanted an “engineer” he went to the Midwest. A NY investment banker said that if a person graduated from ____ that person was guaranteed a job on Wall Street.

Perhaps those college diplomas are like the lights set out by the Pacific Island Cargo Cults? Doesn’t the split between the perceptions of the small privates, like Williams and the public institutions like Maryland, as cited, tell us that there is much much more than what one gets from a measure of what happens in classes.

Perhaps the rise in the Early or Middle College/high school programs and the increasing number of CC’s offering bachelor programs are making these issues visible, particularly when we see the rise is adjuncts in the traditional institutions?

What, indeed, will happen to the College Board when someone says that the king has no clothes? Can Peter Pan save Tinkerbell by misdirection? Please repeat, “I do believe in testing” It’s wonderful to see the timely appearance of Sokal’s new volume, “Beyond the Hoax”

tom abeles, editor at on the horizon, at 2:10 pm EDT on May 20, 2008

Compared to what?

“Try to make it real compared to what?”

That old Charlie Parker (?) line was followed by an explative in the song.

The toaster I bought stopped working after 8 months. It was warranted for a full year(yippee skippee) but the company didn’t honor the warranty.

We seem to receive a lot of students fromK-12 schools or homes where they don’t honor their warranties either.

HOW FAR can a college take a particular student? Community colleges are mostly open-enrollment, even when students come in with learning or perceptual difficulties, andmost CCs take those students to a much higher functioning level.

What scale are the “watchers” using to measure learning? From disfunctional, drug-addicted, beaten down, dyslexic, apathetic, dis-affected — to a job at McDonalds?

If the Dupont, Rothchild, Roosevelt, _______(fill-in-the-blank) child can go from “average student” in 12th grade to a “slightly above average” college graduate in four years; able to manage a family business — isn’t that a success?

What if the 1650 SAT 12 grader zips through undergrad. in 3.5 years, finishes his or her MBA in record time, and lapses into a deep depression after only two years at ENRON.

Our fault? Read the fine print on that human being warranty, the K-12 warranty, the college warranty, the grad. school warranty, or break out your copy of Emotional Intelligence (or maybe the Diagnostic and Statistical Manual of Mental Disorders?).

Even in a forum such as this, nobody can agree to evaluation (outcomes assessment) criteria & procedures. What happens when millions of unique, eccentric individual human students are the “object(s)” and learning is the “subject” of evaluation? Complicated, eh?

600 students can receive the same lecture or “teaching activity” in the same lecture hall or online on the same network, and they score on a (somewhat) normal curve; some “A"s, some “F"s, most scores in the middle. Maybe teaching isn’t nearly as important as all the factors happening in a student’s life?

Some students apparently do a much better job of educating themselves (than do others) using our institutions and instructors and libraries and computers as their tools.

Dr. F. Gump, at 2:15 pm EDT on May 20, 2008

Education, Authority, and Student “Outcomes”

Perhaps the most important educational outcome is usually ignored in these discussions, as it apparently was at the subject conference. Put simply: Is it *what* students learn, or *how* they learn that’s most important? This is a rather subtle problem, because, in order for the process to be most effective, students and teachers alike must believe that the “what” is most important. The “how” is thought to be administrative detail, important insofar as it affects the “what” (e.g., small classes vs large classes; lectures vs socratic dialog, etc.).

But arguably and, I would say, on evidence, it is the “how” that most reliably selects for and/or causes the most important educational outcomes in the great majority of instances. An employer knows, or should know, something important about a job-seeker who has completed high school with a comparatively decent grade average: This person has reliably “come to work” (was “present” at school) for years and has accommodated to supervisory authority by carrying out assigned tasks and submitting to regular formal evaluation by supervisors. The diplomas from high schools whose graduates do not predictably exemplify these traits are not worth as much as those from high schools whose policies enforce these criteria. (Meaning, truancy is treated seriously, classes are orderly, and “social promotion” is not practiced.)

A college graduate is, or should be, reliable in other ways: This individual has attended classes without supervisory sanctions being applied, and without parental oversight; he/she has performed complex tasks (e.g., term papers) without benefit of continous supervisory direction and oversight; and he/she has negotiated a complex departmental and divisional organization over a period of years with minimal day-to-day direction and oversight. The diploma of a college that challenges its students and grades accordingly is worth more than the diploma from a college that graduates students who mainly hang out for a few years and bestir themselves but slightly, with few if any consequences for their lack of commitment.

To slightly over-generalize, the properly educated high school graduate is trained to the requirements of industrial discipline; the properly educated college graduate is trained to the requirements of bureaucratic discipline.

Of course, when educators and students think of school as merely going through the motions, the authority relationships may be compromised to the point that “graduation” doesn’t tell us much about the self-discipline and reliability of the graduate. It’s akin to voting in a presidential election: It’s impossible for anyone’s vote to affect the outcome, but if most people thought that way, they wouldn’t vote at all, let alone thoughtfully and responsibly.

Rod Bell, Adjunct Professor at College of DuPage, at 3:35 pm EDT on May 20, 2008

GREs, SATs

The “empirical” economist should know that SAT and GRE scores are poor predictors of college success. But perhaps that speaks more to the nature of what passes for “empirical” in that discipline (ie, other people’s datasets).

JF, at 5:15 pm EDT on May 20, 2008

We Have No Clue

I am perpetually amused by debates like this one, illustrating how little we academics know about what we’re doing. Take just one word, “diversity.” Few arguments about this subject fail to mention that the “diversity” of higher education institutions renders difficult, if not impossible, any attempt to compare outcomes among institutions.

So how do we characterize or measure institutional “diversity"? We have no clue! Several years ago a colleague and I conducted a little mail survey if what institutions expected their graduates to know and be able to do, as demonstrated perhaps by their stated general education goals and standards. There were about a hundred institutions in the sample, ranging from major research universities to community colleges. Guess what? Their student learning goals were almost all identical, regardless of institution type. “Oral and written communication” was first, followed by “critical thinking,” whatever that is. So, superficially at least, we all agree on basic goals. How come we can never agree on how to determine whether we’ve reached them or not?

Don Langenberg, Chancellor Emeritus at University System of Maryland, at 6:35 pm EDT on May 23, 2008