The SAT 'at War With Itself'

SAN DIEGO -- When American high school students take an SAT that is an hour longer than it used to be, and that includes a writing test many top colleges ignore, Richard Atkinson may be the man they have to thank.

Atkinson is the former president of the University of California. When he announced in 2001 that he was recommending that the university system stop requiring the SAT of applicants, he got the attention of the College Board in a way that other critics of the test never could. The prospect of losing all of those University of California applicants led to all kinds of changes in the SAT (and succeeded in keeping the university among the institutions requiring the test).

In a speech here Wednesday at the annual meeting of the American Educational Research Association, Atkinson said that while the changes had resulted in "significant" improvements, he was decidedly unimpressed with the results. He said that the essential criticism he made in 2001 -- that colleges need measures of achievement and knowledge, not some sense of students' aptitude -- was still valid.

He argued that high school grades and the SAT II (the subject matter tests) could give admissions offices the information they need far more than the SAT I (the main exam, which most people just call the SAT). Atkinson was particularly critical of the "critical reading" portion of the SAT -- and said that it appeared to serve no real purpose, and that it was "remarkable" (and not in a good way) that adding a new test and an hour's time to the SAT had failed to improve its validity.

The remarks were significant for all kinds of reasons. Not only was Atkinson the person who set off the changes in the SAT, but the University of California is now moving in the opposite direction that he endorsed -- eliminating the subject test requirement and keeping the SAT. Atkinson's criticism was also notable because of his background and perspective. Some critics of the College Board are generally opposed to standardized testing. But Atkinson said he strongly believes there is a need for standardized testing. And he's spent years studying the subject, in research on cognition and as past chair of the Board on Testing and Assessment of the National Academy of Sciences.

Atkinson's talk was based on a larger paper he co-wrote with Saul Geiser, who has conducted extensive research for the university system on admissions issues. The paper, "Reflections on a Century of College Admissions Tests," is available on the Web site of the Center for Studies in Higher Education.

In his talk, one of Atkinson's themes was that the underlying flaw of the SAT is that it was designed to measure student aptitude, and remains so, long after the College Board removed "aptitude" from its name. Atkinson said that there is a much higher validity to tests based on actual knowledge learned in courses, and that -- grade inflation being what it is in high schools -- admissions officers genuinely benefit from a national tool to compare students boasting A's in calculus, chemistry or French at high schools where an A may mean different things. (Despite those concerns, Atkinson stressed his view that grades in college preparatory courses are the single best way to predict college success.)

Prior the reforms of this decade, Atkinson said, the SAT was known for "trickery" and "esoteric analogies" that encouraged students to try to learn test-taking skills. He applauded the elimination of the much-mocked analogies section and applauded especially the addition of the writing test to the SAT. Not only does it test something that students actually need to do in college, Atkinson said, but it sends a message to high schools to take writing instruction seriously.

But while backing these changes, he said that the SAT was "at war with itself" because it tells different things to different people. Atkinson noted that the writing test -- the part of the SAT of which he is most fond -- is ignored by many colleges, where educators don't believe it encourages the best student writing. So while the College Board tells Atkinson and like-minded people to focus on the writing test, the College Board tells admissions officers elsewhere that they still have the "old SAT."

And even with the improvements, he said that the SAT remains designed to theoretically measure potential, which he said is very difficult to do well, rather than what students have actually learned. (The ACT, the College Board's main competitor, tends to boast that it is more focused on classroom instruction, and Atkinson applauded its roots in that belief, but offered what for him was clearly a damning criticism in that he said that the ACT has become more like the SAT over the years.)

Fundamentally, Atkinson said, the College Board "wants to have its cake and eat it, too," producing tests that really measure learning (he mentioned Advanced Placement tests and the SAT II tests), while still encouraging millions of students to prepare for an aptitude test of limited value. "The fundamental question is: what is the SAT measuring?" he said. The answer (aside from writing) is that it is testing things that are "remote from what students encounter in the classroom."

Atkinson also directly challenged the spin that the College Board placed on last year's validity studies of the new SAT. The studies found that the new test was as successful as the old test, and the College Board declared that a major success. Atkinson said that if you overhaul major features and add time to a test, success would be increased validity, not the same validity.

The irony in Atkinson's presentation was that the University of California is moving in the opposite direction. The university is dropping the SAT II tests as requirements, while keeping the SAT I. The move is designed in part based on the belief that more black and Latino students will apply and enroll, although Asian American leaders are urging the university to reconsider its plans because of projections that Asian American enrollment will drop.

In his speech, Atkinson did not mention the university system's actions. But an audience member asked him about it and he noted that he is no longer president, and suggested that those interested in the university's current position on the issue consult the university's Web site.

A spokeswoman for the College Board said that the organization was "gratified that Dr. Atkinson enthusiastically supports our Advanced Placement program and the SAT Subject Tests." But she also defended the SAT I. "Years of independent research and thousands of studies consistently show that combining SAT scores and high school grades is the best indicator of college success," she said. "Virtually every college that does not have an open door admissions policy uses multiple measures when evaluating students. The College Board has always vigorously recommended that SAT scores be used in conjunction with grades, the rigor of courses and other measures. Ultimately, no single measure or test fully reflects a student's readiness for college."

She also said that Atkinson was citing studies of the old SAT and that studies of the new SAT have found that it is in fact "more predictive" than Atkinson gives it credit for. "Colleges can depend on SAT scores as fair measures of college preparedness," she said.

While audience members appeared to have mixed feelings about standardized testing, one college president questioned Atkinson for not going far enough.

Robert Weisbuch, the president of Drew University, said that minority applications surged when the institution ended its SAT requirement. If the SAT reliability remains "so low" and it provides "such a minor help" in admissions, while it "mis-shaped kids' lives" by teaching them about education in a "very mechanistic way," Weisbuch said, why tinker with the SAT at all? "Why don't we just blow the whole thing up?"