What Could a New Admissions Test Bring?

Last week's vote by the University of California Board of Regents was largely viewed as a defeat for supporters of the SAT and ACT. But it was also something else.

Regents voted to establish a new admissions test within five years. If they don't, UC will cease to use standardized tests in admissions. As UC officials admitted, not much is known about the new test, although the California State University system has expressed interest in using it, and other colleges could as well.

Inside Higher Ed asked experts to weigh in on the new test. What could it bring to college admissions? What feature would they most like to see in a new test?

Johann Neem, professor of history, Western Washington University, and author of What’s the Point of College? Seeking Purpose in an Age of Reform (Johns Hopkins University Press)

The history of the SAT -- as laid out in Nicholas Lemann’s book The Big Test: The Secret History of the American Meritocracy -- is about unintended consequences. In the mid-20th century, testing advocates hoped that the SAT would create a Jeffersonian meritocracy. Thomas Jefferson believed that intelligence and virtue were distributed throughout the population. He hoped public schools would, as he put it, rake the best students “from the rubbish annually.” Given that universities were bastions of white elite privilege, SAT advocates argued that standardized tests would open universities up to all deserving Americans. And it, combined with the GI Bill and public tax dollars, transformed higher education as first-generation and minority students entered in large numbers.

But then Lemann’s story becomes ironic. The children of the meritocrats became the new aristocrats. As academic ability, measured by the SAT, became the coin of the realm, parents who had been successful invested time and money to ensure that their kids would replicate their success. Parents with means segregated themselves in school districts with similar parents and invested in test prep so that their children would get into the most prestigious schools.

What to do? The first thing is to accept that any metric will create unintended consequences and, if high stakes, establish perverse incentives. The second thing is to rely on holistic measures of student ability. College is for intellectual development; students must come to college prepared for high-level academic work. But no single test can substitute for the careful evaluation of students’ records, developed over time, in different contexts. If we insist on having tests, the tests should simply assess eligibility: Does a student have the reading, writing and numeracy skills to do first-year college work? There should be no score and no percentiles. Just a simple yes or no.

Mary A. Papazian, president and professor of English, San José State University

Academic preparation for university-level work is important, and the nature of the test to be developed will send a signal to prospective students and K-12 educators about the kind of preparation they will need. I have never been a fan of standardized testing. Perhaps that is the literature professor in me. So I hope that the test will be a combination of standardized responses and written responses. And as I look to the interconnected skills and knowledge that will be needed to navigate the 21st-century university successfully and, even more importantly, prepare for the new challenges in the digital economy we are facing, I also want to stress that it is just as essential that the experience of preparing for and taking such a test allows students from all communities and backgrounds, and especially marginalized communities, to see themselves in this world.

In a general sense, questions that focus on the core skills of literacy and quantitative skills that draw on a range of approaches and use daily experiences that students likely have encountered, whatever their life circumstances, remain essential, so I won’t describe those here. Students also should have opportunities to apply higher-level critical thinking and analytic skills, such as drawing comparisons between and inferences and conclusions from information provided and offering strategies for developing solutions to problems that are described. That information can be provided in both textual and data/numerical form.

At the same time, the new approach needs to appreciate the messiness created by structural inequity and how it affects students from marginalized communities. Questions should allow students to hold contradictory information and move from classic analysis of our world and society to complex understanding of the contradictory structures that too often characterize our world. Written responses allow students to move away from the right answer; they allow for ethical standards and opportunities to illustrate how the moral/ethical imagination informs the lens through which a student might approach a problem. If a student had a chance in the question to reflect on the lens through which they approach a problem rather than be required to use only a traditional lens, it will provide that student with a better opportunity to see themselves in the problem. One way to get at this in a question might be to introduce a concept, ask the student to apply that concept to their interpretation of a piece of writing or data, and then ask them to apply their understanding of that text to their own life experiences.

In this way, the questions allow all students the space to reflect on their diverse life stories and experiences. They could well be drawn from material students have already encountered in their high school curriculum. The approach to the new test should be to help students show what they know and can do, to show their potential. In the development of the test, I would recommend involving high school educators from diverse schools. How do they think this process can bring out the growth mind-set in their students and illustrate their potential to think creatively -- and perhaps differently -- about a complex problem?

Lynn Pasquerella, president, Association of American Colleges and Universities, and former president, Mount Holyoke College

For years, the purveyors of standardized tests such as the SAT and ACT have attempted to address equity concerns using tactics ranging from the application of research findings around stereotype threat (which raises self-doubt and anxiety among those who are negatively stereotyped) to guarding against test questions that presuppose fluency with vocabulary derived from a specific socioeconomic background to introducing an “adversity score.” Efforts aimed at upending bias around race, class, gender and disability status will continue to fall short as long as there is an industry dedicated to training students for test taking that advantages the haves over the have-nots. Modifying existing tests has proven to be an expensive undertaking and presupposes the issue is entirely one of instrument design. Yet, as the Varsity Blues scandal unveiled, the system itself is deeply flawed, and the ideal of meritocracy upon which the tests were developed is a myth.

Quality assessment for the future will entail the evaluation of applied learning experiences centered on student agency, as opposed to asking students questions on a standardized test for which we already have the answers. This approach will require the development of a universal portfolio system that allows students to gather evidence of learning proficiency over time, to choose authentic work that they identify as most representative of their learning gains and to reflect on those assignments in relation to their lived experience. The creation of a portfolio system of evaluation, with scorers trained to assess higher-order thinking, communication skills and the capacity for intellectual growth, is the most promising approach to addressing persistent equity gaps and the growing economic and racial segregation in higher education.

Whitney Soule, dean of admissions and student aid, Bowdoin College

In order to define the features needed in a test, we have to fully understand what it is we are trying to learn from the test in the first place. Because Bowdoin has had more than five decades to review and adjust our methodology without a test requirement, we’ve had time to carefully think about each piece of the materials we require, what it is we hope they will show us about a student’s academic preparation and the interdependence of a student’s disposition toward learning with academic work. Academic achievement isn’t isolated to a syllabus, or a grade, or a test score. Learning is comprehensive, a product of curricular options, motivation to learn, curiosity about self and others, and a willingness to see the interconnectedness of information and experiences. Our assessment process blends an understanding of the academic opportunities and achievement with the personality traits that bring learning to life. In other words, we prioritize the combination of cognitive and noncognitive skills in assessing a student’s preparedness for our school.

And we’re not alone thinking about assessment this way. Independent K-12 schools already have a new noncognitive test tool available to them, called the Snapshot.

This is a great opportunity for colleges to think about what makes a student successful at their schools. It’s one thing to choose to become test optional, but it’s an entirely separate project to figure out what the testing actually provided -- what it corresponded to, or not, elsewhere in the process. Were there gaps in the process, even with testing, that absolutely need to be understood and addressed now without testing?

Walter Kimbrough, president, Dillard University

A creative feature for future testing would look at soft or survival skills. What if future tests could look at critical thinking, problem solving or even logic games like the LSAT does? What if tests measured some form of grit that Angela Duckworth has discovered? It might be a way to identify the genius that exists in students that normally do poorly on standardized tests but have the potential to be great in the right environment.

Xueli Wang, professor of higher education, University of Wisconsin at Madison, and author of On My Own: The Challenge and Promise of Building Equitable STEM Transfer Pathways (Harvard Education Press)

Frankly, I am a bit skeptical of the feasibility of developing or identifying yet another test that is supposed to remedy bias and flaws in standardized tests. The bigger question here is: How closely does the new test -- or any reform to the admissions process -- align with espoused values to advance diversity, equity and inclusion? The suspension of the SAT and ACT is a monumental step toward that, but much is left to be seen with regard to the potential new test.

In an ideal world, such a new test should not merely reflect the current content that students have learned in the K-12 system; it should inspire continuous improvement of a curriculum that embraces culturally diverse learning styles, forms of knowledge and ways in which learning is constructed, among other things. It should be fair, open and equitably accessible in terms of what it tests and how it is administered, which would discourage or eliminate teaching to the test and test-prepping practices that advantage groups from affluent backgrounds.

The key players throughout this endeavor should include both postsecondary faculty and K-12 educators as collaborative partners. Ultimately, new test or not, it represents only one facet of the larger issue of equity and college access, as there are many more structural and societal barriers blocking minoritized students’ educational pathways before they even reach college. UC’s search for a potentially better test, whether it is successful or not, will hopefully yield even more clarity toward transforming admissions policy and processes into a wide gateway for equitable college access, as opposed to a narrow gatekeeper.

Steven Brint, Distinguished Professor of Sociology and Public Policy, University of California, Riverside, and author of Two Cheers for Higher Education: Why American Universities Are Stronger Than Ever -- And How to Meet the Challenges They Face (Princeton University Press)

A real question exists whether any test will satisfy the political constituencies behind the regents’ decision. But let’s assume the regents’ interest in a replacement test is legitimate. The leading options are: (1) achievement tests aligned with the high school curriculum; and (2) tests of analytical and critical thinking that assess students’ capacities to solve “real-world” problems.

The Smarter Balanced Assessment is an example of the first type of test. It was introduced in California in 2014-15 as a measure of college readiness. It is aligned with the Common Core Standards in secondary schools, which California adopted in 2010 (but which have not yet been fully adopted by all school districts). Recent evaluation studies show that Smarter Balanced does not produce better results than the SAT in studies of first-year grades and retention at UC. When it is substituted for the SAT in simulations of UC admissions, it does, however, tend to place a somewhat higher proportion of underrepresented minority students in the top 10 percent of UC admits.

The Collegiate Learning Assessment is an example of the second type of test. The CLA presents students with a document library bearing on the solution of a “real-world” problem such as whether a manufacturer should purchase a component from a subcontractor. In such a case, evidence on safety, cost, alternatives, company balance sheets and testimonials are presented to students to evaluate. Students are required to build an argument about what should be done. The CLA is a more interesting test and reflects the skills that many leading employers look for in new recruits. However, underrepresented minority students do not tend to do as well as white and Asian students on the CLA, which probably dooms it in the context of contemporary higher education politics in California.

Belinda H. Y. Chiu, author of The Mindful College Applicant: Cultivating Emotional Intelligence for the Admissions Process (Rowman & Littlefield)

I would love to see the University of California Board of Regents and institutional leaders take this opportunity to set forth a more holistic approach that takes into account students’ academic readiness and social and emotional intelligence. Evidence suggests that engagement, optimism, perseverance and curiosity are correlated with performance at the tertiary level. Integrating these indicators into the admissions process sends a strong signal for all stakeholders of a student’s education (including the student) to help break the cycle of teaching to the test and allow students to focus on their own growth, learning and exploration.

Of course, the tough question is how. Some thoughts, in no particular order, that might be implemented alone or in tandem:

• Portfolio-based assessments, whereby students can show their trials and triumphs by submitting material beyond a few essays that speak to their growth, interests and potential. Most art and design schools already require submission of a portfolio in addition to academic records. The Coalition for College has a virtual “locker” for students to keep a multiyear record of their work and projects. Some may argue that portfolios would encourage Instagram-worthy curation, so it would be important to broaden the scope and length of time covered and triangulate the sources of information, as well as to emphasize that the portfolio should reflect a celebration of one’s mistakes, experiments and curiosities -- a demonstration of growth rather than a sanitized facade.
• On-the-spot essay prompts, where students are required to respond with a bigger-picture, interdisciplinary approach. For example, nearly every SAT subject has some connection to climate change, but no single subject can solve the crisis alone. Students can then better showcase their analytical and problem-solving skills rather than their test-prep capabilities.
• Performance-based assessments, where students work in small teams or otherwise demonstrate their capacity to collaborate and think creatively and laterally. The AAALab at Stanford run by Professor Dan Schwartz is developing innovative approaches using game-based, adaptive learning assessments.
• Virtual third-party interviews, where students don’t have to try to sell themselves to one institution, and the institution doesn’t have to sell itself to the student. This isn’t meant to replace the beloved alumni interview. However, it may give admissions officers a more neutral perspective and relieve students of the pressure to contort themselves. Some may argue that this favors "extroverted" or "polished" students. Yet companies like InitialView that conduct live interviews have demonstrated that it is not the polish that comes through, but the student's authenticity.

Integrating tools like these into the application process may allow admissions officers to get a better sense of a student's ability to tackle complex issues, growth mind-set and adaptability -- skills essential at and beyond university. And it’s an invitation to honor wisdom from multiple lenses and encourage the lateral thinking needed in the “new normal.”

The college admissions process can never be stress-free, and this is an exciting opportunity for reimagining and innovating how we learn.

Robert Schaeffer, interim executive director of FairTest: The National Center for Fair and Open Testing

Over the 30-plus years that FairTest has led the test-optional admissions movement, we have lived through at least five "comprehensive overhauls" of the SAT and the more gradual "evolution" of the ACT. Based on the data, neither process has significantly altered the fundamental conclusion: no standardized test is as strong -- or as fair -- a predictor of undergraduate success as is an applicant's high school record.

The probability that UC faculty can -- in just a few years -- develop a new instrument that is superior to what huge testing companies have produced over several decades is slim. Even if they somehow design such an assessment, there would face huge implementation hurdles, such as how to make applicants from outside California take a state-specific exam. As a number of UC regents on both sides of the debate stated in the course of Thursday's daylong debate, the likelihood is that the University of California system will have no testing requirement in place at the end of this process.