Graduation rates?

With all the demands for more higher-ed funding — when is Congress going to demand that colleges post their graduation rates?

What is the effect of additional funding into systems that fail to produce graduates 50%-65% of the time?

Does anyone care?

L.L., at 8:00 am EST on February 26, 2008

Basic Premise Not Established

I’d like to challenge the creators of this new assessment to share the data they’ve used to establish that all or most law schools claim that ALL of their faculty are good at everything. Indeed, I’m under the impression that aside from some anecdotal puffery that the creators have found on web sites, most consumers of education are well aware that there’s a dynamic relationship between teaching and scholarship. That’s why some schools give teaching relief to great scholars, and scholarly relief to great teachers. The creators of this new assessment must be aware of this, but their basic premise for some reason completely ignores this facet of how schools realistically build their faculties.

Richard, at 10:30 am EST on February 26, 2008

For the record, The Princeton Review also reports law school rankings in its annual “Best 170 Law Schools” book and on its site. Like its college and b-school rankings, Princeton Review’s law school rankings are uniquely based on its surveys of students who rate and report on their experiences at their schools. TPR surveyed 18,000 law school students for the rankings in the current edition published Oct 2007. Among the many ranking categories it reports is “Best Professors” — based on how surveyed students rated the teaching ability of their profs and their accessibility outside class. The #1 law school on this list currently is Boston Univ. and # 2 is U of Chicago. The full list of top 10 law schools in this category and 10 other ranking list categories in the book and info on the rankings is at www.princetonreview.com.

Jeanne Krier, at 12:40 pm EST on February 26, 2008

That project is a farce.

It’s not surprising that faculty (and the 1% of elite students would want to be faculty some day) view the quality of faculty as the measure of a good law school.

But law school applicants don’t. They look at the USNWR rankings. So do law firms that are looking to hire law students. And the USNWR rankings are driven by the LSATs of the entering students.

The Green Bag asserts (without any support) that top scholars make the best teachers. O rly? What do the studies show?

Not surprisingly, the studies show that there is no correlation between the two. Some top scholars are great teachers; some aren’t. Some profs with no scholarship records are terrific teachers; some aren’t. So when they rate schools by recent scholarship they are rating schools by a criterion that matters to almost no law students.

What do students care about? In the absence of good data about bar passage, debt, job prospects, and starting salary (which law schools are loathe to reveal, especially outside the top 14), students use the next best proxy: professional prestige as measured by the USNWR rankings. If Green Bag really cares about the students, as opposed to helping the professors play an inside-academia game that matters only to other professors, then Green Bag can focus on data that matter to students.

bernardooboyle, at 1:20 pm EST on February 26, 2008

Quality as value added?

As long as we all pretend to admire the emperor’s finery, it matters little whether we’re talking about his elegant jacket, his trousers or his hat, and employers will pay top dollar for his photos. Those employers then have their staffs fill in the obviously missing elements in the pictures and carry on as though there is no alternative. Every ten years or so (read “Cramton Report (1979)” “MacCrate Report (1992)") a group reports that in fact the emperor has no clothes beyond his underwear. There follows a period of moaning and lament, a couple of suggestions for improved undergarments and arguments over whether they’re really necessary, since the emperor is attended by only the finest tailors.

Climbing out of the analogy for a moment, what is it that law schools should be good at? Bernard (Bernardo?) suggests a list, half of which depends not on product but on perception (job prospects and starting salary), both of which probably correlate highly with admissions exclusivity and suggest that the principal function of highly selective schools is mere screening. “We’ll cull the brightest ones, entertain them for three years, charge them accordingly, and turn them over to you for training.”

How about a value-added standard? Which schools move their students further from some standard measure at the outset (the LSAT?)to a standard finishing method (the bar exam?)? Which schools turn out lawyers, ready to add value to their employers, rather than law graduates in need of legal training? And which schools do more with less?

Start ranking schjools that way and see where employers and students go.

Andy Starkis, Asst. Professor of Law at Massachusetts School of Law, at 4:10 pm EST on February 26, 2008

You folks are talking about two different things. Employers will only probably care about graduates’ skills, and not care about value-add. But students need to know the value-add of each school, so that they can decide which schools are worth paying higher prices.

No Sucker, Author, No Sucker Left Behind, at 5:50 pm EST on February 26, 2008

What about a ranking of the actual skills of the students — ability to do legal research and writing? And how are schools that offer tenure or tenure-like benefits to their librarians going to be treated? Librarianship is a different animal than teaching (for one, the units available for a librarian to teach are lower, and most “teaching” done by librarians comes at point-of-need rather than at easily quantifiable times) — will those schools (which often turn out students who can actually conduct legal research, unlike the vast majority of law students) be penalized?

Librarian, at 1:40 pm EST on February 27, 2008

Andy,

If that’s your thesis we’re not really in disagreement. But notice what a student still wants: what I’ll call first tier data on bar passage, debt, job prospects, and starting salaries, and, in the absence of that data, she wants second tier data, which is USNWR rankings.

The USNWR rankings drive the law firm’s macro decisions about hiring. It’s how they decide the grade and class rank cutoffs for each school, how they decide which schools to go to, and how to spend their recruiting budget dollars. Firms also consider geography, the schools their own lawyers went to, and on rare occasions a particular subject matter specialty at the school, but the dominant factor that swamps all the others is USNWR rankings. USNWR rankings define the landscape. Everyone in the hiring business knows it. No one with first hand knowledge doubts that at all.

As for the recent success from MSL in that tournament, I am happy for that outstanding performance. I bet those students worked hard and were thrilled. By the way, when you compare your performance to those elite schools, I’m sure you’re aware that in mock trial competitions the less elite schools often dominate. That’s a very common occurence.

And it’s true that local DA offices often look closely at mock trial performances when hiring. But ask yourself this: would the MSL students trade those regional mock trial wins for higher USNWR rankings? You bet they would. And for good reason.

bernardooboyle, at 8:45 pm EST on February 27, 2008

Peter,

Thanks for that link. In response to my claims, the author of that post makes three arguments. On the first, the author alters my claim and then attacks a straw man. On the second, the author lacks knowledge and is factually incorrect. On the third, the author confuses the issue and is factually incorrect. Let’s take them in order. On the first claim, I acknowledged that faculty and a small number of students would “view the quality of faculty as the measure of a good law school.” Note my phrase “the measure,” and grant me a touch of obvious hyperbole for the 1% figure. I asserted that for the other students (the overwhelming majority), the operative measure of a good law school is its USNWR ranking. In response, the author re-writes my claim as if I said that 99% of students have no interest in the quality of faculty. He says that I “claim[ed] that only 1% of applicants to law school are interested in faculty quality.” That’s a straw man. Obviously that’s not what I said. I said that the huge majority of students do not view faculty quality as “the measure” of a good law school. That’s not to say they have “no interest” in the topic. To be clear, for most students “the measure” of a good school is, and shall remain, the USNWR rankings. (Those students will also look at debt loads and geography when choosing which school to attend, but that doesn’t change what they consider to be “the measure.") On the second claim, about whether law firms make use of the USNWR, I claimed that law firms make extensive use of USNWR and that USNWR rankings “define the landscape.” I claim this on first hand knowledge. In contrast, the author claims no first hand knowledge. He doesn’t say, for example, “I’ve been there and I know myself that we and other firms made no use of the USNWR rankings.” Instead, he cites what he says are “a lot of anecdotes” that are not shared with us. To add verisimilitude, the author sprinkles in dated cliches like “coffee room banter” and “once in a blue moon,” as if he spends a lot time in the coffee room with fellow members of the law firm hiring committee, bantering about their internal processes. But he doesn’t. The author also claims that firms have internal rankings, but he does not show that if such rankings really exist they are anything other than echo chambers of the USNWR. (It is also true that law firm will recruit locally and at schools where their own lawyers attended, but will use different cutoffs if those schools have lower USNWR rankings.) I’m not particularly motivated to argue this point as if it were a matter of rhetorical debate. Why would I? I know the reality. For those who are interested in investigating, speak with your friends in law firms or with the career development people in law schools. Ask them if law firms use the USNWR rankings “only once in a blue moon” and only for occasional “coffee room banter.” You should also ask them if law firms make extensive use of USNWR rankings and if those rankings define the landscape. Ask them if firms use the USNWR rankings to determine class rank and grade cutoffs at various schools. Ask them what rankings the law firms are referring to when they make frequent use of phrases like “top ten” and “second tier.” On the third claim, the author alters my argument and also makes a factual error. My claim was that the USNWR rankings were “driven” by the LSATs. In response, the author claims that the LSATs are not “weighted” the most. Those are two different things. To take an extreme example for purposes of illustration, suppose I developed a weighted variable forumula to decide what car to buy, and decided to most heavily weight the factors of “number of wheels,” “presence of cup holder,” and “power windows.” Suppose I gave a lesser weight to horsepower. It’s likely that horsepower would take on an importance far greater than its weighting might suggest and that close to 100% of the variance in my rankings would be driven by horsepower.So what about LSATs? The law school deans commissioned two quantitative scholars to study the USNWR rankings, and as part of that study they analyzed the variance in the rankings. They concluded that “90% of the overall differences in ranks among schools can be explained solely by the median LSAT score of their entering classes.” (The study is about a decade old, but the formula hasn’t changed that much.) So it’s hardly surprising that two other quantitative scholars recently concluded that “student Law School Admission Test (LSAT) scores have assumed (albeit inadvertently) an importance far greater that their current, direct 12.5% weighting.” So my third claim is amply confirmed. And, moreover, LSAT scores can drive other factors such as bar passage and (over time) reputational scores. But there is yet more. LSATs also drive the rankings in the sense that law school admission committees are obsessed with increasing the LSATs of their entering class. They often “game” their reported LSAT scores for just that purpose. Applicants, too, game their LSAT scores (in the legitimate sense) by test preparation courses, and then tend to measure the schools by their USNWR rankings (which, as we already know, are 90% explained by LSATs). It’s a circular process.

Peter, I trust that clears up any confusion on the matter. If you wish to read the studies for yourself, you can find the law deans’ study at aals.org, and the other study is by Andrew Morriss and William Henderson.

Now that I’ve addressed those issues, I want to return to my main point. The Green Bag study on recent scholarship will measure something, but it won’t measure “teaching.” Nor will it provide the measure by which law school applicants decide what is a good law school. It will be most useful for academics to see which schools and professors aren’t publishing much. In certain circles, that study may be highly valued. I guarantee you that it won’t cause even a ripple in undergrad pre-law circles. And even within law schools, when the students want to measure teaching, they will consult the schools’ evaluations and the word of mouth from other students.

bernardooboyle, at 1:00 pm EST on February 28, 2008

Peter,

My weak argumentation has failed to convince you of the truth! We are both undone! Somehow I will manage to recover, but it surely won’t be easy.

best,

Bernardo

bernardooboyle, at 1:50 pm EST on February 28, 2008

Andy,

Again, I am not putting down MSL students and I do congratulate them on their success in that tournament.

My point was that notwithstanding the relatively common occurence of that kind of feat, law firms make pervasive use of USNWR rankings to make their hiring decisions. It seems that you agree with that as a descriptive fact, but it also seems that you are decrying that while I am sticking to a descrptive account.

You make an additional claim: “elite schools don’t really do much for their students other than trade on the widely-held myth of quality. The key quality sold to employers is the quality of students selected, a quality those students brought to the institution, not one acquired there.”

I find that too broad to be accurate. As you know, firms use the USNWR rankings to determine what grade cutoff to use at particular schools. Necessarily, then, the firms are also measuring something that’s happening within the schools. Also, many elite schools do offer very useful training. These days they all teach research and writing skills. A few elite schools have the kind of successful mock trial team you have at MSL. Other elite schools have upper level courses taught by practitioners who know their subject matters in great depth. Some elite schools have strong clinical programs. Some have strong records placing their students in externships that develop their students’ skills. All of those things are also true at some mid-level and some lower-level schools. My point is that it goes too far to say that elite schools don’t do much for the students besides screen applicants for quality.

bernardooboyle, at 4:45 pm EST on February 28, 2008

Dynamic Rankings

Take a look at www.lawschoolpower.com for dynamic rankings that allow the user to weight factors as he/she sees fit. I believe this is a great way to take a more personal perspective when investigating law schools and finding a perfect match. There is also an admissions calculator.

Anon, at 5:05 pm EDT on June 12, 2008