You have /5 articles left.
Sign up for a free account or log in.
The idea of non-cognitive admissions reviews appears to be taking off -- at least in regard to the kind of information sought in letters of recommendation. Grades and standardized test scores -- which reward cognitive skills -- remain the dominant factors at most institutions with competitive admissions. But starting in professional schools and now expanding to undergraduate admissions, new systems to measure other kinds of skills and qualities are moving from being theoretical to playing actual roles in admissions decisions.
The Educational Testing Service is this year starting its first pilot in undergraduate admissions with a system designed to focus letters of recommendation on non-cognitive qualities that may not be sufficiently recognized in other admissions criteria. The system is a version of the Personal Potential Index, which ETS added last year as an option on the Graduate Record Examination for those applying to grad school.
The PPI, as that system is known, is passing a key milestone itself in the coming admissions system, as the business school of the University of Notre Dame is going to require the PPI as the standard system for letters of recommendation for its M.B.A. program -- becoming the first academic program to use the PPI to admit an entire class.
Also this year, the Law School Admission Council has unveiled its own version of a PPI for letters of recommendation -- giving law schools the option of adding a standardized, non-cognitive factor to their admissions criteria.
The idea behind the PPI is to make letters of recommendation more reliable and to also have them focus on non-cognitive qualities that may be extremely important to the chances of student success, but may not be reflected in grades or test scores. Students who use the PPI -- to date available only for graduate and professional school applications -- have faculty members or others evaluate them on six factors: creativity, resilience, communication, planning and organization, teamwork and ethics. Testing by ETS has found correlations between these qualities and subsequent success. Further, and of considerable interest to many admissions officials who worry about racial and ethnic gaps in standardized test scores, the PPI does not produce such gaps.
David G. Payne, vice president and chief operating officer of ETS for college and graduate programs, said in an interview that one reason undergraduate colleges have expressed interest in the PPI is that they want to identify more qualities -- including non-cognitive qualities -- that will encourage retention of a range of students.
Payne said that under the contract ETS has with the College Board to administer the SAT, the College Board had the right of first refusal on the version of the PPI that was developed for use in undergraduate admissions. But the College Board declined to use the PPI, so ETS is proceeding with plans to offer the service by itself -- and it could be used by applicants who take the SAT, the ACT or no test at all. During the next year, a group of colleges will be testing this PPI in their admissions systems.
Over the long term, Payne said that he sees potential for the program to have a significant effect on college admissions. Some of the students "most likely to flourish" will do so not because of the attributes measured by the SAT, he said, and this program could give admissions officers the evidence they need to admit more students without top standardized test scores. He said that there will still be value in traditional measures of cognitive skills, but that in selecting among those who have those skills, colleges will have more of a basis for making choices.
The Collegiate PPI, as it is being called, has six factors (similar to those on the graduate version) on which reviewers rate those seeking admission to colleges: critical thinking and problem solving, motivation and work ethic, ethics and integrity, persistence and resilience, leadership and teamwork, and communication skills. As with the PPI for graduate admissions, there are statements related to these qualities on which those offering recommendations will rate applicants. In this year's pilot, ETS has signed up a small number of private universities (which it declined to identify). Payne added, however, that ETS wants to expand the pilot and would welcome additional institutions -- public and private alike, and with a range of demographics.
The College Board has for several years been talking about how it could add non-cognitive measures to its admissions services, but passed on the chance to use the new ETS service. A spokeswoman for the board declined to say why it did not work with ETS on its new system, saying only via e-mail: "The College Board is very interested in non-cognitive predictors, and other types of admissions tools. We are currently researching various non-cognitive options, including ETS’s PPI, which we think very highly of. At this time we have no specific future plans to share."
Business School and Law School Admissions
In a major advance for the original PPI (for graduate and professional school admissions), all applicants to the University of Notre Dame's M.B.A. program will be required to use it this year.
Brian Lohr, director of M.B.A. admissions at Notre Dame, said that the decision was made out of a feeling that traditional letters of recommendation were providing less and less valuable information. "We found that the letter of recommendation only really became a value add when it was negative, and 99.9 percent are going to say the person is the best thing since sliced bread," he said.
In contrast, the dimensions identified in the PPI "are really important to us," he said.
Applicants are just starting to learn of the new system for submitting letters of recommendation, he said, and reaction has been positive. "They feel that the more complete a picture we can get, the better," he said.
ETS has been playing up the PPI in its push of the GRE into the M.B.A. admissions market, where it is taking on the dominant test, the Graduate Management Admission Test. Lohr said that Notre Dame viewed the PPI as separate from the GRE, and that applicants are free to submit either GMAT or GRE scores -- as long as the PPI is submitted as well.
The Law School Admission Council, which sponsors the Law School Admission Test, is trying a similar approach this year as an option in law school admissions. The council has created its own program (independent from ETS) in which letters of recommendation could be replaced by a Web-based tool in which evaluators rank applicants on six factors -- including non-cognitive factors. The council's six factors are: intellectual skill, personal qualities, integrity and honesty, communication, task management and working with others.
Wendy Margolis, a spokeswoman for the council, said that the option has just been sent to all law schools, with the expectation that some will start offering applicants the choice of using the system this year.
Is the Change Significant Enough?
Critics of traditional college admissions, who have long said that too much emphasis is placed on test scores, had varying reactions to the news that ETS is expanding its work on measuring non-cognitive attributes.
Robert J. Sternberg, provost of Oklahoma State University and a psychology researcher whose work focuses on testing and education, has long advocated that colleges keep standardized tests, but add other measures to measure non-cognitive skills. He said that "it's really positive that they are moving to include non-cognitive measures, that they are looking at more than the SAT."
Sternberg said that it has been clear for years that traditional measures account for only some of the difference in academic performance of students and that non-cognitive factors also play a role, so these shifts are long overdue. But he questioned whether they go far enough. The evaluations of non-cognitive skills that ETS and the law school group have developed focus entirely on the views of those asked to send recommendation letters. Colleges could get much of the benefit by just being clear in asking those submitting letters to focus on the issues that the ETS system evaluates, he said.
A more ambitious approach, he said, would be to have applicants themselves submit evidence of their non-cognitive skills. When he was a dean at Tufts University, before moving to Oklahoma State, Sternberg worked to do just that. Tufts now has optional portions of its application that are designed to highlight creativity beyond test scores. For instance, one optional question states: "The human narrative is replete with memorable characters like America’s Paul Revere, ancient Greece’s Perseus or the Fox Spirits of East Asia. Imagine one of humanity’s storied figures is alive and working in the world today. Why does Joan of Arc have a desk job? Would Shiva be a general or a diplomat? Is Chewbacca trapped in a zoo? In short, connect your chosen figure to the contemporary world and imagine the life he/she/it might lead."
And the optional portion also includes other formats that may play to some applicants' strengths outside of traditional essay writing. Students may, for example, submit a YouTube video about themselves or they can "use an 8.5 x 11 inch sheet of paper to create something. Blueprint your future home, create a new product, draw a comic strip, design a costume or a theatrical set, compose a score or do something entirely different."
Just looking at non-cognitive skills from the view of those recommending applicants is "really a baby step," Sternberg said, although he's glad to see it taken.
Robert Schaeffer, public education director of the National Center for Fair and Open Testing, said he found it telling that the College Board rejected a chance to include the new ETS service in the SAT. Schaeffer noted that the College Board has from time to time shown interest in Sternberg's research and in other projects that reflect individuals' varied talents and experiences that might not show up in an SAT score -- but has backed away.
"Now they have turned down an opportunity to include the more nuanced information the PPI could provide in conjunction with SAT scores," he said. "This pattern suggests that the College Board's highest priority is preserving the market position of its flagship product -- the SAT -- not providing admissions offices with detailed, relevant evidence from multiple sources which could improve their decision making."
