Nov. 13, 2006

Coaching and Lasting Out New SAT

It took the College Board years to admit that coaching could help students taking the SAT. At the board’s annual meeting Friday, officials admitted that the new writing test — a key part of the new and expanded SAT — is coachable, with significant gains possible for those who would otherwise receive low scores.

Looking for a job?

See all 312 new postings

Browse all job listings:

Faculty: 2,352
Administrative: 2,117
Executive: 155
FEATURED EMPLOYERS

At the same session, board officials suggested that they aren’t likely to please those who have been pushing the board to let students — many of whom complain that the SAT is now too long — take the different parts of the SAT at different times. The College Board had previously announced that it was reviewing the idea, which high school counselors have been pushing. But the head of a College Board committee said Friday that the only approach under consideration was letting students retake the SAT’s parts in separate sittings, and that everyone would need to take the entire SAT at least once in one sitting.

Coaching on the Writing Test

The impact of coaching has always been a sensitive issue for the College Board. Because coaching services cost money (and some of the more elite coaching services cost a lot of money), a coachable test raises questions about fairness to low-income students. In recent years, the College Board has acknowledged that coaching works for some students, but has repeatedly said that coaching services overstate the impact of their programs, and that most gains are modest.

On the College Board’s Web site section for test takers, the organization expressed the hope that the new SAT would be ‘less coachable” than the old one. The site repeats past statements that many students who are coached show little or no gain, and says that some testing companies — which the board does not name — are hurting students by telling them to memorize an essay that they can write down regardless of the essay question. This is bad advice and will hurt students, the board says.

But at the board’s annual meeting, Wayne Camara, vice president for research and analysis of the College Board, described research that recently found that coaching — even short-term coaching — does have a major impact on those with poor writing skills. For the research, six graduate students were assigned to put together a coaching program by signing up for a bunch of coaching services, and then developing a coaching program based on what they had learned.

A group of college freshmen were then divided in two: One group went through a nine-hour coaching program led by the graduate students and the other students were the control group. The students took the writing test before and after the coaching, and the coaching had a significant impact on those who did poorly the first time around — increasing their scores by an average of 3 points on the 12-point scale used on the essay.

Camara also said that the study had the students write two essays that are of the sort that college freshmen actually write (not the timed essay with prompt that is on the SAT). Coached students who were poor writers at the beginning did better on the actual essays and not just the SAT, Camara said. As a result, he characterized the coaching for the writing test as “not a bad thing,” in contrast to other coaching, which he said is more likely to teach test-taking skills than meaningful knowledge.

Asked whether the research might in fact point to a serious problem by giving a new advantage to wealthy applicants, Camara said that even if that is true, the coaching could be replicated in public schools so that low-income students could also benefit. In this case, the coaching is providing “good instruction,” so he said he couldn’t oppose it — even if it was only available to some test takers.

Other experts on testing had different takes on the significance of the new research.

Andy Lutz, head of research and development for the Princeton Review, said that the College Board is correct that the writing test is coachable. “The College Board is right — and surprisingly honest for once,” he said. Lutz said that the Princeton Review commissioned a third party study on this that won’t be out until next year, but that all the evidence he is seeing from Princeton Review counselors suggests that the new writing test “may be the most coachable portion of the SAT.”

He echoed the views of many college officials who have decided not to use the writing test in saying that its approach is flawed. “It’s a bad way to measure writing skills — a 25-minute essay on an esoteric subject.”

Many of the SAT essay questions are philosophical in nature — the debut question, for example, was on whether majority rule is always right — and some think that the nature of the prompts contributes to their coachability as students are taught a mix of tactics to formulate good essays.

Ed Colby, a spokesman for the ACT, said that testing service had not conducted research on the coachability of its new writing test. But he said that ACT intentionally picked non-esoteric topics for essays so that student responses would be more natural. Recent topics have included high school dress codes and the best time for the start of a school day.

Robert Schaeffer, public education director of the National Center for Fair and Open Testing, said that it was irresponsible for the College Board to just accept the way its tools give more help to those who are wealthy. “If coaching works, how is the SAT a common yardstick when an admissions office cannot tell whether an applicant’s score reflects intensive coaching or no preparation at all?”

Is Fatigue a Problem?

With the addition of the writing test, the SAT is now 3 hours and 45 minutes — and from the first time it was given last year, students, parents and counselors have been complaining that the test is too long.

When several hundred counselors wrote to complain, College Board officials said that they would study their recommendation that students be allowed to take separate parts of the SAT at separate times. Camara said that the College Board’s research has found that the length of the test does not affect students’ ability to do well. The board compared the rates at which students provide incorrect answers or skip questions — and those rates are constant throughout the test. If fatigue was causing problems, Camara said, the rates wouldn’t have stayed constant.

Previously, College Board officials have said that the organization’s SAT Committee was studying the possibility of letting students take the test in separate sittings. John Barnhill, director of admissions and records at Florida State University and chair of the SAT Committee, said that College Board officials had determined that every student needed to take the test in its entirety at least once.

Barnhill noted that each SAT includes some questions that are being used to plan for future SAT versions and aren’t part of the student’s actual score. If students could take the different parts of the SAT at separate sittings, he said, they might not answer those beta questions and that would hurt the test.

The SAT is considering letting people who take the test multiple times select only one part to take again, he said. For students who take the SAT more than once, many colleges allow students to take the highest score they received on each part of the SAT, so a student may be evaluated on a mathematics score from one day she took the test and the critical reading score from another day.

Currently, such students must retake the entire SAT, although Barnhill said that many such students will focus only on the part for which they are trying to raise a score and will ignore the rest of the test, not worrying about the low score they will get on other sections.

The College Board is studying whether such students might be spared having to go through the entire test a second (or third) time, he said. But logistics may make that difficult as well, he said, since currently students don’t know the order of the test.

FairTest’s Schaeffer said that the College Board’s approach to the issue showed the “fundamentally inconsistent” way it examined policy questions. “Limiting the opportunity for separate sections of the SAT only to those who can take the test multiple times gives an additional leg up to students from upper-income families who can afford to pay the College Board multiple registration fees,” he said.

The question the College Board should ask is whether the scores are equally valid, he said. “If taking sections separately on a re-test does not damage the SAT’s predictive value, why does the same reasoning not apply to a student’s initial administration of the exam?” he asked.

— Scott Jaschik

Want it on paper? Print this page.
Know someone who’d be interested? Forward this story.
Want to stay informed? Sign up for free daily news e-mail.

Advertisement

Comments