Law Schools Flagged for Job Data

The first audits of the employment data that law schools report about their recent graduates have generated concern among watchdogs, with a series of reviews finding several deficiencies that raise questions about the class of 2015’s reported outcomes.

Most notably, a review of 10 randomly selected schools found that half had missed a compliance benchmark for the documentation they are supposed to keep on file when reporting key metrics like whether their students are employed 10 months after graduation and whether they are working in a position that required them to pass the bar. Schools were flagged for not being able to show documentation to support important parts of reported employment data, or if investigators found evidence key pieces of employment data were incomplete, inaccurate or misleading.

Other reviews found issues at a substantially smaller percentage of schools related to handling documentation or posting required information online.

The audits, performed at the behest of the arm of the American Bar Association that accredits law schools, are not final. ABA leaders say they do not have a hard timeline for when the reviews will be completed, but indicated that the issues uncovered tend to be clerical in nature and were not instances of "gross misreporting" or "attempts to manipulate."

That hasn’t stopped critics from wondering whether the audits reflect widespread problems in the employment data law schools use to attract prospective students, however. Such problems would come years after allegations of deceptive job placement data led to a political firestorm and after the ABA made significant changes to the data-reporting requirements to which its schools must adhere. They would also come shortly after the ABA took a tough stance on two law schools' admissions standards in the wake of it coming under fire this summer from the federal panel that oversees higher education accreditors.

Still others cautioned that auditing is a major step for law graduate employment data, and that it will take some time for the ABA to teach schools what it wants from them.

The initial audit results were delivered in an Aug. 5 memorandum to the Council of the Section of Legal Education and Admissions to the Bar, the ABA’s accrediting arm. The memo, from Deputy Managing Director Bill Adams, said that contractor Berkeley Research Group conducted three types of initial reviews: a website compliance review, a random school review and a random graduate review.

The random school review picked 10 schools and reviewed all of their graduate employment files to determine if deficiencies existed. The random graduate review took a sample from all graduate employment files at the 205 ABA-approved law schools to search for deficiencies. The website review searched each law school’s website to see if it posted a current employment summary report and compared the data in that report to employment information the school gave to the ABA. Schools could be subject to additional scrutiny if issues were found.

Institutions fared worst in the random school review, which required 95 percent of a school’s graduate employment files to comply with rules that they hold documentation supporting employment information reported to the ABA. Files were to be deemed deficient if they lacked documentation to support key parts of reported data or if investigators found credible evidence of incomplete, inaccurate or misleading data.

Just five out of 10 reviewed schools posted compliance rates of 95 percent or more. A sixth school missed the mark by a hair, posting a compliance rate of 94.8 percent. Two more came in at 86 percent, and a final two were found to have compliance rates in the mid-50 percent range. One of the lowest scorers appeared to have created supporting documentation after it was told it had been chosen for audit, the memo said.

The random graduate review involved 382 files from 156 schools. A total of 16 schools were marked as having a potential problem with missing documentation or items. Half of those were deemed to have minor issues unlikely to warrant additional review. The other half were deemed to possibly need heightened review, and one appeared to have created documentation for its files after it learned it was to be audited.

Finally, the website review noted one school had not posted its 2015 employment report on its website as required. Auditors noted discrepancies between data online and data reported to the ABA at four schools. Three schools’ discrepancies were classified as being “only by one person in one category of their report.” More minor discrepancies were noted at the other school, but Adams’s memo deemed it to not be a large number.

Berkeley Research Group was holding follow-up talks with schools about potential issues, the memo said. Most schools had issues related to documentation, and it was not yet clear that any school had misreported data, it said.

“Our documentation requirements are quite specific and some think complex, so it is reasonable that in the first year of this review, there may be confusion over what we require,” Adams wrote. “The two schools that appear to have created their documentation after the fact raise more serious problems, but they may also be able to explain that what we perceive is not accurate.”

The issues identified tend to be small things like which boxes to check on forms or where to store information, according to Barry A. Currier, managing director for accreditation and legal education at the ABA.

“If the underlying information turns out to have been accurate, which is what we are finding, that’s good news,” Currier said in an interview. “Then, getting them to do the paperwork in exactly the right way is our major concern, and we’re just checking and doing some education.”

There are no plans for releasing the names of law schools flagged for issues, Currier said. The schools flagged did not fit into any particular rankings class, region of the country or type of institution.

“The only way it would be reported is if a school turns out to have misrepresented or been so uncooperative that we end up issuing some sort of a censure or sanction,” Currier said. “But other than that, everything else we do, with the exception of those kinds of big-picture process things, is confidential.”

There are also no plans for expanding the audit pool -- even for the random school review, which found issues at half of the schools scrutinized. Currier had no projected date for finishing the audits, adding that schools have been cooperative and that the process has been a learning experience for both the schools and his office.

“We feel like we’re actually reasonably far ahead of the curve of what other accrediting agencies are doing,” Currier said. “This is not required by the [U.S. Department of Education].”

The issue of gainful employment has, however, cut across higher education sectors. The Department of Education released regulations last week intended to allow debt discharges for student loan borrowers who were victims of an institution’s fraud or misrepresentation.

The ABA audit’s initial findings drew interest from law school observers -- and differing levels of concern. The website violations are a significant concern, said Kyle McEntee, the executive director of Law School Transparency, a Georgia nonprofit focused on legal education policy. The schools posting 50 percent compliance rates are troubling as well, he said.

But McEntee is not as worried about the quality of data law schools are collecting as he once was. It’s important that the audits are being done, he said.

“I think law schools have done a really good job of changing their practices over the last six years,” McEntee said. “But it’s so important the accreditor take this seriously and take that paper trail very seriously. I expect it’s going to take a year or two for schools to work the kinks out.”

A lawyer who was a key player in prior legal disputes over employment figures had a harsher reaction to the audits. New York City-based Strauss Law PLLC was a leading actor in the battle over law school employment data, filing suits for plaintiffs suing schools for allegedly inflating hiring figures for their graduates. Firm founder Jesse Strauss was not surprised that the audits found compliance issues.

“The schools will do whatever they can to game the stats, since they are graduating too many lawyers for too few jobs,” he said in an email.

Disgruntled bloggers were critical as well. The blog Outside the Law School Scam labeled the audit results shocking and called for an expansion of the number of schools audited.

“Otherwise, we will be left to wonder whether a school's employment survey results are really just lies in highly granular form, a whole bunch of numbers that represent not actual jobs for recent grads so much as a school's success in hoodwinking the ABA and its prospective students,” it said.

It is difficult to measure the significance of the audit findings without more context, said Brian Leiter, a professor at the University of Chicago Law School and director of the Center for Law, Philosophy and Human Values there. Institutions failing the audits because they are not familiar with reporting requirements would be different from institutions failing because they are trying to conceal poor employment outcomes for their graduates, he said.

The audit results could have been better for a law school sector that has gotten a bad reputation in recent years, Leiter said. But the ABA has improved reporting requirements. Now it is time to follow through and make sure schools are adhering to them, he said.

“You would have liked to have seen a very high rate of compliance,” he said. “We didn’t get that.”