Settling the Score

The College Board has overhauled its scoring methodology for the majority of Advanced Placement exams over the past two years. On Monday, the nonprofit published on its website the first public acknowledgment of the shift in the way it scores more than three million assessments each year.

Prior to 2022, AP exams in each subject area were scored according to standards set every five to 10 years by a panel of 10 to 18 educators—about two-thirds of them college professors and one-third high school teachers—who used “individual expertise to estimate the percentage of AP students who should earn each AP score,” according to a statement from the College Board. The new methodology, called evidence-based standard setting (EBSS), analyzes much larger data sets with input from hundreds of faculty members who review the exams and recommend scoring distributions based on their students’ performance in comparable college classes.

In an interview with Inside Higher Ed, Trevor Packer, head of the AP program, said the College Board had been looking to revamp its scoring process for years but only developed the capacity for the new methodology after 2019, when it introduced AP Classroom, a digital platform for course materials that instructors can use to track student progress and administer practice exams. That allowed the College Board to gather large amounts of metadata comparing student exam performance to grades in AP classes, Packer said, and adopt what he called “a more scientific, data-focused methodology.”

Most Popular Stories

Most Popular

“[EBSS] guards against variations in panelists because it keeps the standards tied to specific skills and content knowledge demonstrations that we can maintain over time,” he said. “It also won’t fluctuate according to college grade inflation.”

In 2022 and 2023, the College Board used both its traditional panel-setting process and the new EBSS method side by side to compare scoring outcomes; when the two were at odds, officials relied on the EBSS score, according to a College Board spokesperson. This year they switched the vast majority of AP tests to EBSS only; five remaining exams continue to use standards set by panels but will transition to EBSS in the near future.

The change in score-setting methodology explains a significant increase in average student scores on many AP exams over the past few years, which have led to headlines like “Are AP Exams Getting Easier?” and inflamed concerns that the College Board is enforcing less rigorous scoring policies to broaden the appeal of AP courses.

Packer said the increase in average score distribution for some AP exams is actually a course correction for subjects that had long lagged behind others in terms of the student success rate—the number of test takers who earn at least a 3, the minimum score most colleges require to grant academic credit. He said the goal was to bring all exams to between a 60 and 80 percent success rate.

“We would never position this as a recalibration … these AP scores have moved around for decades,” Packer said. “We’re using evidence and data to put scores where they should be.”

Not everyone in the AP community is on board. John Moscatiello, a high school teacher in New Jersey and founder of Marco Learning, a company that supports school districts building AP and literacy programs, dubbed the scoring shift “the Great Recalibration” in a critical blog post (and accompanying TikTok posts) that has made the rounds in AP teacher Facebook groups and admissions blogs. He said that even after seeing the official explanation on Monday, he’s unclear on how the scoring change is a response to objective data.

“This doesn’t seem to be reasoning your way to a conclusion based on evidence; it seems like rationalizing your way to a foregone conclusion,” Moscatiello said. “It may be that this is the correct thing to do, and that scores have been misaligned for years, but I think there are legitimate doubts about this method and why it always seems to produce the same result of maintaining and raising scores rather than lowering them.”

Swinging Standards

Since the College Board began implementing EBSS in 2022, nine AP exams have seen significant increases in student success rates: AP Literature, U.S. History, U.S. Government, World History, European History, Macroeconomics and Microeconomics, and two science tests, AP Biology and Chemistry.

For some of those courses, the upward swing has been so sudden and steep that it’s raised eyebrows among observers across higher education. Score distributions before 2022 are not accessible via the College Board’s webpage, which includes links to scores for 2023 and 2024. But the data is still available on PDFs that were at one time posted to the College Board website, which can be found through a simple Google search.

The success rate for AP English Literature, for example, skyrocketed the year officials switched to EBSS, from 43.9 percent in 2021 to 77.9 percent in 2022. For AP U.S. History, the success rate increased by 16.7 percentage points in 2024, the first year EBSS was used; rates for AP Chemistry went up by 21 percentage points between 2022 and 2023, also the first year EBSS scoring was introduced in that subject.

The changes are striking at the individual score level, too. In 2021 12 percent of students earned a 4 on their AP English Literature exam, and fewer than 5 percent earned a 5; the next year, during the EBSS pilot, 27 percent earned 4s and 16 percent earned 5s. The inverse is true on the other end of the scale: 22.5 percent of students earned a 1 on their AP Chemistry exam in 2022, but when EBSS was applied the following year, that number fell to just 8 percent.

Packer said scoring for such courses has always been more variable than for the other 26 AP exams on offer.

“In most AP subjects, that success rate has typically been between 60 and 80 percent. But in these nine primarily humanities subjects, that’s been much more volatile … English [Literature], for example, has ranged from a 70 percent success rate to down in the 40s,” he said. “Those were defensible methodologies and scores, but the variation was hard to explain—we had a hard time explaining it to ourselves, let alone to teachers and faculty.”

Packer said he considered the possibility that an explosion in AP test takers had led to a decrease in average scores: from 2012 to 2022, the number of high schoolers taking AP classes grew by nearly 220,000. But the EBSS analysis did not bear that out, Packer said.

“There’s no indication that AP population growth has watered down students’ ability to score 3 or better,” he said. “I wish that was the case—that AP was so widely available now, and all students going to college are taking AP, so you are seeing a much lower-ability population. That’s just not what the data show.”

Rather, it seemed the panel standard-setting process itself was to blame for the decline, which Packer said could be due to the more subjective nature of those courses.

“I have my own hypothesis that college curriculum is less consistent across humanities disciplines than it may be in STEM disciplines,” Packer said. “Maybe that is why the panel-based methodology was more subject to variation over time, as panelists changed and brought their own institutions’ more unique perspectives to bear.”

Some college admissions leaders have supported the scoring shift despite the sudden hike in success rates. Stuart Schmill, dean of admissions and financial aid at the Massachusetts Institute of Technology, said he saw no reason to change how MIT considers AP test results in admissions decisions or in granting college credit.

“I have talked with College Board staff about the updates to the scoring standards and feel confident that the standards are appropriate and aligned with student performance in college classes,” he wrote in an email to Inside Higher Ed.

Of the five most popular AP courses on offer, four saw their success rates surge after EBSS implementation; the other, AP English Language and Composition, has yet to move to EBSS. That has raised concerns that the College Board’s true motives are to broaden the appeal of its fastest-growing AP offerings by making it easier to earn college credit. Jon Boeckenstedt, vice provost for enrollment management at Oregon State University and a vocal critic of the College Board, called the scoring shift “a callous attempt to generate more revenue.”

“I’m very concerned about effectively passing students out of entry-level classes and getting them placed into more advanced classes they might not be ready for,” he wrote in an email. “For an organization that claims to be student-focused, [the College Board] seems to be completely oblivious to the downstream effects of their business and revenue enhancement decisions.”

Packer was adamant that the scoring changes have nothing to do with increasing AP course enrollment.

“If the data suggested we needed to drop the scores, as panels have done in the past, then we would have … Now, we have evidence saying the scores should be higher in these nine subjects, so we raise them,” he said. “Self-interest is not at all our motivation, as some may claim.”

Who’s in the Know?

While the College Board began using EBSS to set AP test scores in 2022, as recently as May 27 its webpage on scoring only mentioned the traditional panel process, according to archived versions of the site on the Wayback Machine.

Packer said the changes were communicated to college admissions leaders, faculty and high school teachers as they happened over the past two years, in “thousands of briefings” with College Board staff and in materials given to AP readers.

“Our priority has been communicating about the standards-setting process and score changes with the higher ed and AP communities … We do this as a matter of transparency and principle,” he said.

But many AP teachers and admissions professionals are only just becoming aware of the changes and have said they feel blindsided by the late notice. In private Facebook groups for AP high school teachers that Inside Higher Ed was given access to, instructors reacted with a mix of confusion and surprise to the news. Some indicated they’d been vaguely aware of changes in scoring over the past few years, but many were nonplussed by the details.

“I’m a high school teacher. I’m at these conferences and briefings they give; I know many other AP teachers who have been, too,” Moscatiello said. “We knew generally that AP exams would be recalibrated, but almost all of us are surprised by this new methodology.”

Wednesday was the start of the annual AP conference in Las Vegas. Packer told Inside Higher Ed he wasn’t sure he needed to address the score changes; he believes the College Board has been transparent with higher ed leaders and high school teachers and is less worried about informing those in the test-prep business. Besides, he has more urgent news to share, like the AP’s impending digitization, which he said he planned to announce in his opening plenary today.

Moscatiello, the high school counselor, is also attending the conference, where he’ll present a panel on AI in Advanced Placement. He said he’s gotten such an outpouring of interest in the new AP scoring process that he plans to devote part of the session to discussing it.

“We weren’t given enough clarity about what will end up altering how tens of millions of dollars in college credit is awarded every year,” he said. “Hopefully that will change soon.”