You have /5 articles left.
Sign up for a free account or log in.
A physician has used AI to create a medical school rankings tool that allows the user to input their preferences as an alternative to traditional rankings.
Photo illustration by Justin Morrison/Inside Higher Ed | U.S. News & World Report
As the backlash against traditional college rankings continues to gain momentum, one physician has used artificial intelligence to create an alternative ranking system for prospective medical students.
Instead of relying on U.S. News and World Report’s widely recognized rankings, which consider factors such as medical schools’ research profiles, acceptance rates and the grades and test scores of their students, the new AI-powered ranking tool allows aspiring doctors to include—and choose the weight assigned to—numerous additional factors, such as diversity of students and cost of attendance.
“I don’t necessarily think the U.S. News rankings are bad. The issue is that they reflect the specific vision of one group and one organization,” said Dr. Brandon Turner, a radiation oncology resident at Mass General Brigham in Boston who designed the alternative rankings tool. “I want to make it transparent what is actually being measured and for people to have the capacity to adjust for things they care about.”
Dr. Brandon Turner is a radiation oncology resident at Mass General Brigham in Boston.
Brandon Turner
Dr. Turner’s idea for his rankings system, which he described in a paper published in JAMA Network Open in July, has been in the works since he was in medical school. But he started questioning the value of traditional rankings even earlier, when he took a course on the sociology of education as an undergraduate at Wake Forest University in North Carolina.
In that class, he learned about the inequities of standardized testing.
“When I thought about the dichotomy—we’re using a tool that we don’t really know is helping us that much, but we do know it’s causing certain students to be excluded from these institutions and preventing them from having the same opportunity I was so grateful to have—I thought we needed to do something about this,” he said.
Dr. Turner kept those disparities in mind during medical school, but finding time to pursue his idea of creating an alternative rankings system was a challenge.
“The skills that you use to do well on standardized tests are not really relevant to most real-world problems,” he said. “There’s plenty of overlap there, but it’s not perfect overlap. And it’s who’s lost when we lose that overlap that’s really the concern.”
Concern about racial disparities in medical schools and in the profession over all has increasingly been the subject of discussions among the medical establishment. A handful of medical schools are working to address student equity gaps but, if current metrics are any indication, the medical schools still have a long way to go.
According to the Association of American Medical Colleges, while 49.8 percent of accepted medical students in 2019 were white and 22 percent were Asian, just 7.1 percent were Black or African American; 6.2 percent were Hispanic, Latino, or of Spanish origin; and 9.4 percent were multiracial or multiethnic.
When Dr. Turner started analyzing the U.S. News medical school rankings and thinking about designing his own ranking tool, access to these schools was one of the things he considered.
“One thing that was obvious to me was that schools rely so heavily on these standardized tests, because they’re evaluated on those metrics for the most popular school ranking in the country,” he said. “If you could change the dynamic there and free schools from feeling like they have to show they have the biggest number possible on these kinds of tests, then maybe schools could think a little differently about what kind of students they think have the most promise.”
Joseph Soares, the professor who taught the sociology course at Wake Forest that sparked Dr. Turner’s interest in rankings, has long criticized traditional rankings as “part of the inequality machine” whose focus on metrics rewards “the richest, oldest, whitest institutions in the country” and aren’t meaningful in showing the value attending one of those institutions could add to a person’s life.
Dr. Turner’s creation, however, cuts through some of those flaws.
“It’s extremely valuable because it provides people the opportunity to select different metrics,” Soares said, noting that rankings differ based on which preferences a user selects. “Data transparency is different from thinking you can rank everybody by one yardstick.”
What Dr. Turner learned from Soares resonated while he was studying for the Medical College Admission Test.
Like all students who apply to medical school, Dr. Turner had to take the entrance exam, and he knew he had to do well—all the top-ranked medical schools boast high average scores among their students.
“I was driving myself crazy trying to get the highest possible score. I spent almost the entire summer studying 12 hours a day,” he recalled. “Having now thought differently about these rankings and these tests, it was obvious how much this was a waste of my time. I wasn’t learning anything new and becoming smarter as a person. My score was getting better because I was getting better at taking that specific test.”
“I knew that part of the reason why schools cared so much about these scores was somewhat as a demonstration of your academic ability, but grades also capture that. I think the real reason these tests are so highly valued is that they’re so highly valued by the U.S. News rankings system.”
But he knew he still had to well on the exam if he wanted people to take his skepticism about testing and rankings seriously.
“People will think you’re just complaining because you didn’t do well on it. But if you ace the test, and then you complain about it, then people will listen to you,” Dr. Turner said. “I thought of that every day I studied. It became my mission to ace this test, so that when I said I thought the test and the way we use it was flawed, I would have some authority.”
Dr. Turner aced the MCAT and started compiling a list of medical schools to which he planned to apply. He admits he looked at the U.S. News rankings, but one of his focuses was finding a program with a strong oncology department that would position him to do a combination of research and clinical work.
That program ended up being Stanford Medical School, which was one of the top-ranked programs by U.S. News at the time. Stanford announced earlier this year that it was pulling out of the U.S. News medical school rankings.
“It is OK to want to go to places that are prestigious,” Dr. Turner said. “I want people to feel like they have choice: you should go to those places because you actually feel like they have an opportunity that will make a difference for you.”
If a program isn’t a good fit, then it doesn’t “matter that much what the U.S. News thinks about the medical school you went to” because, “ultimately, most of my patients don’t really care where you went to medical school.”
Once he started medical school at Stanford, Dr. Turner started putting his idea for an alternative rankings system into code. But the demands of the program slowed his efforts.
“I was starting to lag and lose steam when one of the residents said, ‘Brandon, if you don’t finish this thing and publish it where people can actually see it, it basically never happened,’” he said. “He was right. It has to have a way of reaching people for it to actually get picked up and used and maybe change the way we think about school rankings.”
That conversation pushed Dr. Turner to work through some of the challenges with coding the project, but he eventually put it on hold again to focus on finishing medical school.
Medical, Law Schools Reject Rankings
But a swelling tide of criticism of U.S. News’s metrics for ranking law and medical schools in recent years pushed him to revisit—and finish—the idea. In addition to Stanford, medical schools at Cornell, Harvard and the University of Pennsylvania were among those that dropped out of the rankings earlier this year. Several law schools, including those at Harvard and Yale Universities, also no longer participate in the rankings.
Dr. Turner, who is completing his residency through Harvard, knew this was his moment.
“People are ready and willing to think differently about rankings. I could provide some guidance on how that could look” he said. “Once that happened, I did a lot of sleepless nights and grinded it out.”
Administrators at Dr. Turner’s alma mater have already taken notice of the project.
“Metrics alone don’t tell the entire story of our selectivity,” Dr. Iris Gibbs, associate dean of admissions at Stanford’s medical school, told Inside Higher Ed. “They’re not capturing any of the data to support the things that are really important to students and institutions,” such as the level of financial aid offered, inclusivity and student well-being.
“We’re proud of looking at alternative approaches—like Dr. Turner’s—that may use machine learning,” Dr. Gibbs said. But even with alternative approaches, “we need to be careful that the data that’s used to develop those models are not also reinforcing some of the potential biases that may be in existing data sets.”
Dr. Turner said his machine learning–powered tool is “totally dependent on what you tell it to care about.”
His rankings system features more than 20 categories of factors that may be important to prospective students, including average debt of graduates, board-exam pass rates and percentage of underrepresented minority students. Users can rate the importance of each of those on a scale of 1 to 100. From there, the algorithm ranks the institutions and clusters similar ones into different tiers.
“What I stress to people is that more is not always better. Just because something has the highest number doesn’t explicitly mean it’s better than other schools, but I also recognize people want some sort of rankings and guidance,” he said. Ultimately, though, “You have to know what it is you’re looking for” in a medical program and a future career because “the best school is probably the one you’ll do the best at, which is not something an algorithm will be able to tell you.”
Zach Bernstein, a student at the State University of New York Upstate Medical University, said he used both the U.S. News rankings and Dr. Turner’s alternative rankings system to narrow down a list of medical schools to which he planned to apply.
But what he found most useful about Dr. Turner’s system is its ability to let an individual user decide which aspects of a medical school’s offerings are most important.
“It helps you cut through the politics of the rankings,” Bernstein said. “A lot of the same schools are ranked toward the top, but there’s a lot of schools that have different strengths and fit different needs … You can come here and get answers you wouldn’t be able to find because of the artificial intelligence component that does some of the work.”
Meghana Palaniappan is a senior at Northeastern University in Boston and is still compiling her list of possible medical schools to apply to. She met Dr. Turner last summer while she was a working as a medical assistant at Brigham and Women’s Hospital and he told her about his rankings system.
“One of the most challenging parts of applying to med school is figuring out which school is best for you,” she said. Cost and diversity of the student body are some of her most important considerations. The results showed her some medical schools she hadn’t previously considered but that strongly matched her preferences.
“It helps narrow down the list,” she said. “You don’t have to apply to as many, because all of the ones you’re applying to you know you fit.”
Palaniappan has told her friends about the tool, but Dr. Turner hopes it can reach a wider audience and be adapted for a variety of different academic programs and institutions.
“I would also like to build out a way for people to bring in their own data,” he said. “So the limiting step won’t be what data I’m able to find, but the limiting step will be people’s imagination and ability and access to information they want to see reflected in these rankings.”
Dr. Turner has so far worked on the project alone. He said he’ll probably need funding and technical assistance—something he hasn’t had the time to pursue while finishing his residency—if he wants more people to know about and use his ranking tool. He thinks it could be a useful tool for students exploring all types of higher education, not just medical school.
Prestige Factor
Until the rise of U.S. News’s rankings of colleges and professional schools in the 1980s, prospective students mostly used guidebooks about individual institutions to make a decision about where to apply. But reliance on those rankings can have unintended consequences, said Michael Sauder, a sociology professor at the University of Iowa who co-authored the book Engines of Anxiety: Academic Rankings, Reputation, and Accountability.
In one example, some law schools redistributed resources to give scholarships to high-achieving students in an effort to raise the program’s Law School Admission Test scores—and in turn raise their rankings.
“Schools would tell us that was money they used to use for need-based scholarships,” Sauder said.
And while the backlash to the U.S. News rankings has received attention over the past several years, Sauder said he “hasn’t seen enough yet to think that backlash is going to have a permanent effect on the existence of rankings or the influence of rankings.”
He said prospective college applicants benefit from having information from many different rankings systems. “Rankings are most problematic when there’s only one and everyone’s going to that same one,” he said.
Dr. Turner’s more individualized approach to medical school rankings is especially useful, Sauder said, but making people aware of it is an entirely different challenge.
He recalled a similar website created about 15 years ago that allowed law school hopefuls to rank programs based on individual preferences.
“It offered an alternative to U.S. News, but it couldn’t really compete with the name brand,” Sauder said. “Even when you have these cool alternatives, there’s the problem of getting people to use them.”
Bill McGaghie, a professor of medical education and preventive medicine at Northwestern University’s medical school, has questioned the value of any type of medical school rankings for years, especially because all medical schools must meet tough accreditation standards to remain open.
“It’s a way in which people can assert that one school is better than another. That may be true if you’re concerned about wealth and prosperity,” McGaghie said. “All of the medical schools in the United States and Canada will all provide a rigorous education that will prepare people to take care of patients.”
