SAT Controversy Continues

For months now, the College Board has been saying that it has new measures in place to prevent a repeat of the embarrassing scoring errors on the October SAT. On Thursday, the board released a much-awaited report, which said -- yep -- that the board has new measures in place to prevent a repeat of the embarrassing scoring errors on the October SAT.

The board released the report at the insistence of a powerful New York State senator, who had threatened legal action to obtain it, and to whom board officials promised the report months ago. While some critics of the board said it was good that the report was finally out, the emphasis of the study -- on steps taken and that potentially could be taken to improve scoring accuracy -- bothered many. They said that the report revealed little about how the scoring errors took place or why they took so long to discover and report.

The report was prepared by Booz Allen Hamilton, a consulting firm that found that "the current process [of scoring] is reliable and has prudent controls in place to safeguard scoring accuracy for those marks made in accordance with test directions." The report also found that "operational changes made by College Board in response to the October administration further improved process reliability by introducing scanning redundancy, more frequent scoring checks, an environmental acclimation period to eliminate the effects of humidity, and other safeguards."

While the report largely praised the College Board's system, as improved, it also outlined 16 "secondary risks" that -- while not posing grave dangers to reliability -- could cause problems. These risks include items related to the actual scanning process ("bubble alignment error due to paper manufacturing") and to the "upstream" process when exams are moved and scored (the risk of packages being tampered with prior to shipping). For each of these risks, the report outlined steps that the College Board could take to mitigate them.

In a statement, Gaston Caperton, president of the College Board, praised the report and pledged to deal with all of the risks that have been identified. "The College Board is addressing every one of those risks," Caperton said.

To the extent that the report didn't satisfy critics of the College Board, it wasn't so much what was in the report but what wasn't. The Booz Allen Hamilton study was forward looking in that it emphasized risks ahead and how to deal with them. It didn't contain the kind of analysis many SAT skeptics have been seeking on why thousands of scores were incorrect and how the College Board handled the incident.

Sen. Kenneth P. LaValle, chairman of the New York Senate Higher Education Committee, criticized the College Board Thursday for holding on to the report for so long without releasing it (the report is dated May 26) and for not figuring out why the mistakes happened in the first place.

"The board's lack of accountability in this matter goes to the heart of the arrogance that exists when an organization basically holds a monopoly in a certain market," LaValle said in a statement. "This report still does not answer why such a large number of testing errors occurred on the October 2005 SAT. In addition, the board has not provided an explanation of why it took several months to inform over 4,000 students of inaccurate test scores."

LaValle renewed his call for "strong accountability" for the College Board. Unlike many critics of the College Board, LaValle may have the clout to force changes. He is the legislative father of the "truth in testing" movement in New York State, which led to earlier reforms of the testing industry -- largely against its will.

The National Center for Fair and Open Testing responded to the report by releasing a list of questions that the study did not answer. Among them: What was the root cause of the problem? When did the College Board receive the first request for hand scoring of the October SAT? How many weeks did it take for the board to respond to that request? When did the College Board first discover a broad problem? Why was there a delay between that time and when test takers and others were informed of the problem?

FairTest and Senator LaValle have a history of sparring with the College Board. But the continuing controversy over the SAT also has many admissions deans frustrated. Jeff Rickey, dean of admissions and financial aid at Earlham College, said he couldn't understand why the College Board hadn't released the study earlier, and speculated that it might be because of "their habit of displaying an abundance of arrogance and a shortage of humility."

Rickey offered a suggestion: The College Board should change the name of its yearly meeting of members in November from "Annual Forum" to "Asking Forgiveness."