It's Time for the Talk

This week several admissions experts, through the National Association for College Admission Counseling, released an important report that concludes colleges and universities with test-optional undergraduate admissions yield more low-income students and have similar degree completion rates. Their evidence from case studies of 28 universities informs the burgeoning test-optional movement.

More than 1,000 colleges and universities have eliminated SAT/ACT requirements, and nearly every week universities and graduate programs -- even in die-hard STEM disciplines like physics and astronomy -- are announcing policy changes on the Graduate Record Examination. If your college, university, graduate program or professional school has not discussed whether standardized test scores should still be required or collected in your admissions process, it’s time for the talk.

As professors who have devoted the last decade to understanding the equity implications of admissions, we see Graduate Record Examination use as an important topic in an overdue conversation. We’ve presented our research and led faculty professional development on admissions at more than 40 universities over the last two years, including private institutions like Princeton and Vanderbilt Universities and the Massachusetts Institute of Technology, and public universities like Ohio State University, the University of Illinois and the University of California, Santa Barbara. Through this work and our collaboration with the Inclusive Graduate Education Network, we’re working to equip faculty for holistic, equity-minded graduate admissions.

We’ve learned that a serious conversation about current admission routines is a necessary first step. Graduate admission decisions are typically unenlightened, carried out through ad hoc processes inherited from past generations. Outdated assumptions often rule the day, current data are rarely brought to bear and (surprise!) the racial and gender inequalities that professors say they want to reduce are instead reproduced. And given the decentralized nature of graduate education, there is little, if any, accountability for these results.

Overreliance on GRE scores is a persistent problem, as the test maker Educational Testing Service has acknowledged. In physics, for example, more than one-third of Ph.D. programs filter applicants using a cutoff score, a practice that is in opposition to the ETS Guide to the Use of Scores. Such practice disproportionately --and inaccurately -- brands women, racial minorities and U.S. citizens in the applicant pool as unfit for graduate work. What’s more, meta-analysis of GRE validity studies finds little evidence that scores are associated with, much less drivers of, the outcomes faculty want to see (e.g., research performance, degree completion).

Absent opportunities to take a step back, most admissions systems remain plagued by addiction to quick metrics, unfounded beliefs about what scores signal and anecdotes about great students with high scores and “that one time we took a risk” on someone with low scores.

Half the battle is getting faculty comfortable discussing these matters. Faculty are notoriously loath to change, and Ph.D. admissions committees tend to avoid controversial discussions. This is especially the case on matters related to racial and gender equity, which may require them to confront their privileges and biases.

But what is true in the research -- and which we’ve both noted around the country -- is that faculty who undertake critical introspection can create more equitable systems of review. They can have candid conversations, check their colleagues’ misperceptions and preferences, and learn from one another.

We can also learn from recent examples of robust debate about the role of the GRE. The University of Michigan’s Program in Biomedical Sciences (PIBS) changed its GRE requirements after advocates from both sides wrote white papers and held a public town hall meeting. Just last week, the Astrophysical and Planetary Sciences department at CU Boulder held a facultywide vote that resulted in making physics GRE subject test scores optional to submit. It’s the latest example in a fieldwide movement since the American Astronomical Society issued an open letter to department chairs that “recommends that graduate programs eliminate or make optional the GRE and PGRE as metrics of evaluation.”

These organizations and others have brought context-specific data and social science research to their debates. Decisions to significantly downplay or eliminate GRE score requirements have anchored on several issues:

• Significant score differences based on race, gender and citizenship
• The GRE’s modest relationships with the types of success faculty are most concerned with in graduate school
• The inability of test-reliant admissions processes to consistently identify students who complete the Ph.D.
• Disadvantages to rural and low-income students posed by the GRE’s format and cost
• Graduate schools and the National Science Foundation dropping GRE score requirements for key fellowships
• The growing number of peers that are rethinking the GRE

Recognition of the GRE’s limits evolves into conversations of “what now?” That’s why the discussions we facilitate with faculty are not isolated to standardized tests. Faculty benefit from assessing how all admissions routines have consequences for access and success, and we provide tools for more efficient, equitable, systematic review. We recommend creating evaluation protocols, for example: to mitigate bias, to ensure committees explicitly seek student contributions to diversity and to assess the noncognitive/socioemotional skills that support success in research.

The reflection required to recreate admissions protocols is encouraged by data about application and admission patterns (disaggregated by demographics), as well as questions like:

• How could our admissions practices and priorities better align with the program’s mission? Who is on the committee? Is the leadership sufficiently long-lived to make and sustain change?
• What qualities marked our most and least successful students? How do we select for the former and against the latter?
• How do we assess achievements relative to potential? Should we target the few applicants with the strongest overall accomplishments, or the larger pool we believe can grow and succeed in our program?
• How are we explicitly working to select students who will contribute to the diversity of the department?
• How could our admissions process -- and approach to graduate education, generally -- account for the fact that most nonfinishers leave graduate school for nonacademic reasons?
• To what extent do the application structure and prompts provide the information we need?
• Do faculty receive training in admissions review? How could the review process be reorganized to check biases inherent in graduate admissions?

Making test scores optional can’t be the only strategy for increasing diversity, and if such policy is adopted, systems are needed for assessing applicants who do and do not submit scores. One approach: ignore optionally submitted scores until the very end, thereby judging all applicants, at least initially, on the same information. One Ph.D. program we know had a “reveal party” after making tentative decisions, to see which applicants’ scores were inconsistent with their overall assessment of the file. Another stopped collecting physics GRE scores altogether, and holistically judges academic preparation on grades, coursework and general GRE scores.

At the end of the day, thoughtful discussions will come to different conclusions on whether to require test scores. But failing to even have the talk is a disservice. It allows inequalities enabled by the system to perpetuate, and fails to improve how we identify students with potential to develop into tomorrow’s leaders.