Keep the LSAT

The knives are out for the LSAT, as most people call the Law School Admission Test. The American Bar Association is considering a proposal that would allow law schools to go test optional. The decision hasn’t been made yet, and it would take some time to carry out. And even if law schools are allowed to go test optional, that doesn’t mean that they will. Schools that believe the LSAT is a valuable part of the admissions process would be free to require it. Still, the pressure to go test optional would be immense, especially for less prestigious law schools. And the trend is clear: standardized tests are under fire like never before, and the LSAT is the next battleground.

For some, this is great news. Standardized tests, like government inspectors, real estate appraisers and tax auditors, have never been popular with the general public. Polls consistently indicate support for test-optional policies even when people believe that these tests are good ways to evaluate students. The College Board, which produces the SAT, perhaps the most reviled of standardized tests, claims that “83 percent of students say that they want [the] option to submit scores in college applications.” Still, the trend toward test optional is clear.

Part of this is self-interest: even people who think the tests predict academic performance may not want a requirement imposed on them. And while basically everyone I’ve met believes that standardized tests underpredict their ability, I’ve never met anyone who thinks that standardized tests overpredict their ability. Having a common measure of relevant skills sounds great until that measure indicates that you’re not as awesome as you’d like to be and/or think you are.

Another important factor is equity. LSAT scores vary by race, and the difference is dramatic. A recent report found that the average LSAT score for Black test takers was 142, which is about the 18th percentile, while the average for white and Asian test takers was 153, which is about the 55th percentile. For some, this fact alone proves that the LSAT is racially discriminatory and worthy of elimination. Or, in the words of one commentator, “The LSAT falls short of its egalitarian mission.”

Defenses of the LSAT also fall in line with arguments in favor of standardized tests in general:

• While the LSAT isn’t a perfect predictor of law school performance, it’s a very good predictor, especially in conjunction with grades.
• Having a common measuring stick is an important guard against grade inflation, which is a real danger given the incentive to maintain high graduation rates.
• The LSAT also guards against fraud. The proctored, secure environment verifies that the people taking the LSAT are in fact who they are supposed to be. You might be able to hire someone to write a paper for you, but it’s very difficult to have someone take the LSAT for you.
• Racial performance differences are obviously a cause for concern. But differences alone do not prove that there is anything wrong with the test. Given the inequality manifest in virtually every aspect of society, it would be a miracle if a test could erase every unfair advantage and deliver equal results for everyone. Standardized tests are therefore a reflection of inequality, even though they can magnify it, too.
• Questions are screened for bias, and the performance of every standardized question is monitored for disparate results. This gets complicated, because this procedure, called differential item functioning, doesn’t ensure that each demographic group will get the same score. However, it does screen out questions that are poor predictors of the overall performance of members of some groups even if those questions are working well for everyone else.

In these aspects, the debate over the LSAT mirrors the debate over standardized tests in general. But there’s something special about the LSAT: it’s actually a really good test. I say this from my own 25-plus years of experience preparing people for the LSAT and many other tests, and it’s not just me: the LSAT is widely respected in the test-prep community because it measures important skills in a sophisticated and fair manner. Not everyone agrees, of course, but it’s very difficult to engage with the LSAT without emerging with some level of respect. To do well on the LSAT, you need to:

• Analyze arguments
• Draw sophisticated inferences
• Determine the relevance of evidence
• Identify assumptions
• Spot logical flaws
• Recognize similarities and differences between arguments

Along the way, the LSAT also requires you to comprehend complex texts on unfamiliar subjects and reason abstractly. Does anyone really believe that these skills are irrelevant to performance in law school or the practice of law? Put another way, would you want someone representing you who couldn’t do these things? Again, everyone acknowledges that these skills aren’t all you need for success, but aren’t they part of the picture? There’s a reason the LSAT has remained remarkably similar over the past 30 years. It aims at relevant skills, and it measures them very well.

Granted, I’m not exactly a neutral observer. I help students prepare for the LSAT, among other tests. But I’m no cheerleader for standardized tests. I’ve argued that the current version of the SAT is a blight on education, an enforcement mechanism for a vision of education that makes society worse. I don’t defend the GRE, which tests a mishmash of skills and knowledge that, as a whole, has only the most tangential relationship to the future studies of the customers who are forced to pay more than $200 to take it. The ACT is straightforward but extremely time-pressured, which works out fine for some people but is deadly for others. All the tests that include math impose a vision of math education that forces students to study content that doesn’t interest them and is, in most cases, irrelevant to their goals. If I had the power, I’d substantially revise all of them. But the LSAT is just a better test, and getting rid of it just because we don’t like the results it produces it like fighting high blood pressure by banning the cuffs that tell us when we’re in trouble.

And make no mistake, we are in trouble. The state of our national discourse is evidence enough that we have difficult thinking critically, and there’s evidence that college isn’t doing much to improve the situation. Reading is in decline, and it should come as no surprise that these issues are reflected in performance on the LSAT. Pressures to maintain graduation rates undermine accountability, and we so we need a fair measure of these skills that exists outside of the undergraduate credentialing system. The focus on “drill and kill” and the crowding out of higher-order thinking skills have taken their toll.

So it should be no surprise that we find college graduates with sparkling GPAs from the most prestigious colleges who cannot evaluate arguments or handle complex texts. It’s not really their fault. They’ve done what they’ve been asked to do and endured a punishing slog through mountains of busywork, but they’re unprepared for an environment in which those skills aren’t rewarded.

The LSAT changes the rules, and in a good way. There should be some kind of check on a system that leaves so many students unprepared to think critically. And while we should have sympathy and offer plenty of support for students who have done everything demanded of them and still lack the skills they need, we should also consider the students who haven’t succeeded in today’s academic environment but still deserve a chance to prove that they have what it takes to succeed in law school. Standardized tests offer that opportunity, which is especially important today when so many students are disenchanted and demoralized with their educational experiences.

If anything, other standardized tests should be more like the LSAT. Wouldn’t the SAT be a better test if it measured logical reasoning instead of obscure concepts of Algebra II? The ACT includes questions on ellipses, matrix multiplication and factoring polynomials. Those concepts are relevant for some, but isn’t it more important that college students be able to analyze arguments and determine whether information is relevant? Making these tests more like the LSAT would be a step in the right direction, though we would need to brace ourselves for the results: when we test critical thinking skills, we probably won’t like what the results tell us.

Still not convinced? Let’s game out the system if law schools become test optional. People will celebrate, and lots of applicants who would have been unlikely to attend law school will be admitted thanks to their high GPAs and glowing recommendations. But what happens then? For starters, there will be an explosion of applications, which is good for law schools, which look more prestigious when acceptance rates drop. But applicants will adjust by sending out more applications, just as they do in today’s college admissions game. Without the LSAT, it will be harder to tell if your application will be competitive, which makes admissions decisions more unpredictable, and so people will adopt the “shotgun” approach, which raises the costs for everyone. Without the LSAT, GPA becomes even more important, and that’s a problem because even if your GPA is great, so is everyone else’s. Having good grades will then become less of a credential and more of a requirement. As in high school, students will be even more deterred from dabbling with new subjects in college because doing so might damage their GPA. This happens now, but it will get worse when GPAs become even more important. In these ways and others, the admissions process will be different, but it won’t necessarily be better.

And, of course, we should consider the long-term consequences. There’s a correlation between LSAT scores and bar-passage rates, and if you look at the bar-passage rates of schools with lower LSAT scores, the results are sobering. So are we really doing people a favor by admitting them to law school and giving them six figures of debt when there’s good reason to believe that they won’t pass the bar? And even if they do, would you want someone with subpar reasoning skills representing you? Granted, the bar exam and legal practice aren’t everything. Some people, like me, for example, graduate from law school and don’t want to practice law. But surely it’s a problem if large groups take on immense debt and then can’t practice law. Figuring out who is at risk is a good thing, and while we’re at it, we might use what we learn about the lack of critical thinking skills to inform some much-needed educational reform. Tests like the LSAT should be part of that process, because recent events have made it clear that we all suffer when we can’t evaluate arguments, draw logical inferences and spot assumptions. These skills are essential in law school, the practice of law and life in general.