Moneyball for Tenure? Not Until Academia Is a Team Sport

When I saw the title, “Moneyball for Professors,” I thought, Uhh…that doesn’t sound so good.

When I read the article’s log line, “Using analytics to improve hiring decisions has transformed industries from baseball to investment banking. So why are tenure decisions for professors still made the old-fashioned way?” I thought, That really doesn’t sound good.

Reading the article by Erik Brynjolfsson and John Silberholz of MIT’s Sloan School of Management simultaneously made me more and less concerned that prominent academics at an elite institution would float such an idea.[1] 

The stuff that bugged me, a kind of unquestioning belief in the idea that because we can aggregate and analyze data we should aggregate and analyze data, and having done so, should substitute it for human judgment is such a common refrain, it ceases to surprise.

The unspoken, but omnipresent belief that academia functions as a meritocracy is equally predictable coming from big time, big thinker-type academics.

The idea isn’t a “Moneyball” approach to tenure or hiring, really. Relying on a data set confined to “54 scholars who obtained doctorates after 1995 and held assistant professorships at top-10 operations research programs in 2003 or earlier,” the authors “prove” that their method selects a group of scholars with a higher productivity rate in getting published in what they call “top” journals,[2] than the individual decisions of the tenure committees.

According to the authors, this method is superior to more straightforward citation counts because it’s built on what they call “The Academic Dual Network,” which quantifies both the citation network of a scholar’s work, and the scholar’s co-authorship. Essentially, the method quantifies how connected both the scholar and their scholarship is to other people who are “leading” scholars in their field.

I am not a data scientist, but doesn’t it all seem a little circular? A little insular? Is it difficult to foresee that such a process may reinforce and indeed accelerate any existing structural inequities?

The authors admit other limits, that they’re predicting research “success,” and research success isn’t the only criteria for tenure. It’s interesting that they don’t also question the limits to their own definition of research “success,” but as I say earlier, this is to be expected.

So no, it’s not a Moneyball for tenure. It’s an example of what has become in the words of Mike Sharkey, a higher education data analytics consultant, the “obligatory Moneyball reference,” a way of stoking excitement for “data-driven” analysis by invoking a well-known and easily graspable narrative of data predicting “success.”

But the obligatory Moneyball reference obscures the true implications of how data intersected with baseball to change the game. Moneyball is not strictly a story of predictive analytics, but is instead about identifying “market inefficiencies.” Certain aspects of playing baseball -- namely how often you get on base, and slugging percentage -- tended to be undervalued, therefore if one team decided to privilege those numbers over others, they would have an advantage.

I suppose, in theory, you could look at the MIT researchers’ “Academic Dual Network” and see it as an identification of a market inefficiency as compared to citation counts, but this overlooks an important and obvious distinction between Moneyball and education. As Sharkey points out, in baseball, there is no ambiguity about what it means to “win.”

In academia, however, things aren’t so simple. Getting tenure may be an example of individuals “winning” inside the system, but, and this may also seem obvious, academia is not a team sport.

I’ve written previously about how I believe there is a disconnect between “tenure as principle” and “tenure as policy.” Tenure as principle -- a way to ensure sufficient security, stability, and freedom to do the work of scholarship, teaching, service, and governance -- seems an unalloyed good to furthering the goals of the institution. Even if we can’t precisely identify what it means for an institution to be “winning,” at least in theory, tenure empowers faculty to pursue individual paths, which we hope add up to a successful whole.

But “tenure as principle” has become “tenure as policy,” not something to be bestowed when one meets the criteria for tenure (no matter if it’s subjective or objective), but something available only to an elect few. The very existence of tenure lines has become an administrative prerogative. The labor which we associate with earning tenure -- teaching, scholarship, service -- is being done by many in the faculty without any possibility for tenure. Because of a structure of tenure as policy, this divide is essentially arbitrary, determined by what category you are hired under, and having little to do with the labor you do.

This divide between tenure as principle and tenure as policy can exist because there is no widespread agreement on what it means for an institution to be winning. I’m not even sure too many places are even considering the question beyond numbers like enrollment and their balance sheets. Winning is merely keeping the doors open, which often means exploiting their laborers, and charging students increasingly higher tuition.

This exploitation both impacts contingent faculty, who labor without the possibility of tenure, and are therefore in many ways excluded from the game, as well as tenured faculty, who are left behind to tend to the larger needs of the institution which require tenure. Tenured faculty are “collapsing” on the job because, to extend the sports analogy, there aren’t enough players on the field. 

It would be great if we could apply the true principles of Moneyball to higher ed, if we could figure out what contributions to the cause are being undervalued by the systems, but first, we have to agree on what winning means.

If we believe in “tenure as principle,” my gut says a pretty good proxy for how well an institution is doing would be the percentage of faculty who are eligible for and then earning tenure. I’d like to see the data scientists start to explore some of those questions.

All those individuals “winning” would probably be good for the team overall.

But before we start worrying about a better quantified system for determining hiring or tenure, there’s some other questions to answer first.