Plus for AP and Personality

Colleges might gain much more information about the likely success of undergraduate applicants to science and technology programs by giving added weight to success on Advanced Placement tests, and on personality traits (different ones for males and females) that may hold back some students, according to a new study.

The study, published in The Journal of Educational Psychology (abstract available here), tracked 589 undergraduates at the Georgia Institute of Technology to see whether they persisted as STEM majors. A much larger study of Georgia Tech students -- by some of the same researchers and forthcoming in Teachers College Record -- found similar results for the predictive value of AP scores on top of traditional measures (high school grades and SAT or ACT scores). The new study finds that while traditional measures explain about 25 percent of the variation in student performance, the additional consideration of AP scores and personality traits allows for explaining 40 percent of the variation.

The authors are three psychology professors -- Philip Ackerman and Ruth Kanfer of Georgia Tech, and Margaret Beier of Rice University. (The paper acknowledges that Georgia Tech attracts applicants and enrolls students with superior records in math and science, compared to students at many other institutions, but the authors believe the findings would either hold true -- or might even have more impact -- at other institutions.)

In an interview, Ackerman explained that the findings could suggest that colleges consider AP test scores in a different way. Further, he said changes in AP testing patterns allow for a change. Early in the program's history, AP was primarily for high school seniors, so test results wouldn't have been available until after colleges had decided whom to admit. Now it is common for many students to take more AP courses before their senior year than in their senior year. As a result, high schools and colleges have developed ways to reward that behavior, but they may be doing it the wrong way, Ackerman said.

Many high schools give bonus points on grade point averages (and many colleges allow such boosts) for AP courses, so that an A in an AP course might be worth 5.0 instead of 4.0 and so forth. But Ackerman said that the correlation with student success in college STEM courses isn't just enrollment, but is mastery of the material sufficient to earn a high test score. So if colleges want to know who will succeed in college STEM programs, they should be looking at the AP test scores, not just registration for an AP class.

"By focusing on performance on AP and similar kinds of exams, testing applicants for what they know in addition to their abilities and prior performance appears to be able to boost our predictions of their success," he said.

Beier said in an interview that the findings show the value of "domain knowledge" can identify more than "your general ability and aptitude" on whether one is likely to succeed.

But both Beier and Ackerman warned that there is a potential down side to colleges favoring applicants with high AP scores. "I think they are good predictors of success in college, but if you have colleges looking more at those scores, it disadvantages students who haven't had the opportunity," she said.

Ackerman said that while there has been much work to expand access to the AP program, it is still the case that better-off students attend high schools with more AP opportunities. He declined to speculate on whether the results said anything about the validity of the SAT. (The study that has just been released was financed by Georgia Tech, but the study forthcoming was supported in part by the College Board, which is the sponsor both of the SAT and the AP programs.)

Some educators have worried that many students are trying to take too many AP courses, and Ackerman said that the new paper should not be used to encourage such a frenzy. He said that the predictive value of AP test scores came from their average, not their quantity, so someone who has a high average from three AP courses was as likely to do well in STEM in college as someone who achieved the same average on 10 AP courses.

The Personality Impact

The study also examined personality traits of students and found different linkages for male and female students who started off as STEM majors but then shifted to other programs. The women who moved away from STEM fields were those who did not have a "self concept" of themselves as scientists -- in other words, those who didn't see themselves as scientists. Notably, self-concept doesn't necessarily have anything to do with ability in science, so some of these women likely are highly capable in STEM.

For men, the personality trait associated with moving away from STEM was that they lacked traits of "mastery and organization." This would be men, Beier said, who reported that they lacked time management skills and "couldn't get things done."

Beier said that these findings suggest not only that admissions officers might seek out those without those traits, but that educators could try to reach students early so they are less likely to have those traits. For instance, she said that colleges might work on study skills with male students and educators might do more work early on with girls who are talented in science to show them that they can pursue STEM careers.

Robert Schaeffer, public education director of the National Center for Fair and Open Testing, a longtime critic of the SAT and other standardized tests, said that he was not surprised by the findings. Many researchers, he said via e-mail, have found that "non-cognitive indicators can be powerful factors in forecasting undergraduate academic performance. This study demonstrates quantitatively that such traits add significant predictive value. It should now be clear that exclusive reliance on conventional measures, such as G.P.A. and ACT/SAT test scores, ignores important evidence in evaluating applicants. The data about differential impact of scores from specific AP exams may also have implications for the admissions process."

Trevor Packer, who leads the AP program for the College Board, said he had not seen the new research.