Anonymous Comments, Unmasked Bias

Several recent studies suggest women have a harder time than men making career inroads in economics: female economists take longer to have their papers accepted by journals, for example, and they get relatively fewer tenure-track jobs. A new working paper is notable, then, in that it appears to shed light on some of the attitudes and stereotyping working against them.

The paper, by Alice H. Wu, a recent graduate of the University of California, Berkeley, who is currently a research specialist at Princeton University and in 2018 will begin doctoral studies at Harvard University, investigates gendered language used on the popular Economics Job Market Rumors forum.

Using methods from text mining, machine learning and econometrics, Wu analyzed more than a million posts on the website over two years. Over all, she found that conversations between anonymous parties on the forum become significantly less academic and professionally oriented -- namely, more personal and skewed toward physical appearance -- when women are mentioned.

Using her own classification system, Wu counted the number of academic or professional words and personal or physical words in each post. On average, posts specifically about women contained 43 percent fewer academic or professional terms and 192 percent more terms about personal information or physical attributes.

Source: Alice Wu

On gender-related posts, words most strongly associated with women are mostly inappropriate, Wu says. “The occurrence of these words in a forum that was meant to be academic and professional exposes the issues of explicit biases in social media.” Top examples of such words are "hotter," "hot," “attractive,” “pregnant,” “gorgeous,” “beautiful,” “tits,” “lesbian,” “bang” and “horny.” By contrast, the list of top words associated with men includes references to sexual orientation but also "philosopher," "keen," "motivated," "slides," "Nordic" and "textbook."

It's well-known that such terms are frequently used to describe women online, but the site Wu studied is mostly used by economics graduate students and Ph.D.s, not a general audience.

Analyzing comment threads, not just individual posts, Wu also found that conversations focusing on women show similar patterns. Posts also tend to deviate more drastically from being academic or professional if the prior post is related to women.

Confining her analysis to discussions of high-profile economists (based on the Research Papers in Economics index), Wu also found some evidence that women receive more attention than their male counterparts.

Wu says in her paper that gender stereotyping can be subtle, making it difficult to measure. Moreover, she says, people tend to hold back on their beliefs due to a desire to appear politically correct. So the anonymity of the Rumors site “provides a natural setting to study the existence and extent of gender stereotyping in this academic community online.”

Results suggest the need “for changes to maintain an inclusive online environment for everyone in the academic community,” Wu wrote. “The casual setting of this online forum cannot be an excuse for gender stereotyping conversations, and the freedom to express one’s opinions anonymously should not be abused to create a sense of isolation, which can be discouraging and harmful to the academic and professional development of all genders.”

Wu said Saturday that she first heard about the forum from friends who were curious about comments about prominent economists, and that she was “shocked” when she saw some of the remarks about women. So she became interested in trying to quantify what was happening.

Asked about how her findings might translate to campus, Wu said she was optimistic that commenters on the forum aren't representative of the field, and that “things will get better as we discuss the issues openly.”

David Card, Class of 1950 Professor of Economics at Berkeley and Wu’s thesis adviser, said he also hoped commenters don't reflect the views of the profession as a whole. But it doesn’t take “a large fraction of hostile colleagues to affect the environment,” he said. Card noted, for example, how the recent remarks of a now former Google employee alleging that women are inherently less suited for work in tech roiled the industry.

“All it takes is one person making disparaging comments to set a negative tone that may cause some young women to opt out of the field,” he said. “In my experience people who assert statements like that are often immune to evidence or rational argument, and envision themselves as superior beings who are more enlightened than the rest of us.”

Erin Hengel, an assistant professor of economics at the University of Liverpool, published the paper on delayed publication timelines for women earlier this year. Specifically, she found that women spend two years getting their papers past peer review and into journals, compared to about 18 months for men. That’s despite her additional finding that women’s papers were more readable both before and after peer review. Hengel said this week that if Wu’s paper “reflects the opinion of even a minority in the field, it suggests female economists still struggle professionally with being taken seriously.”

Coupled with her own findings, Hengel said Wu’s research also suggests women lose valuable work time responding to extra scrutiny. When female economists are called out by colleagues, either in peer review or an anonymous online forum, she said, “they’ll spend more time perfecting their papers and less time writing new papers. This certainly impacts the quantity of their output.” And unless that conscientiousness is appropriately rewarded, via higher acceptance rates (which they’re not), “women are at a disadvantage in the academic job market.”

While economics remains a field with relatively few women, there’s nothing in Wu’s research to suggest that the field is worse (or better) than others in terms of stereotyping of women. Indeed, the internet in general is a harsh place for women, and women professors. An analysis of Rate My Professors, for example, found that words such as "smart" and "intellect" are more likely to be used in ratings of men than women, and "genius" is more likely to be used to describe male than female professors in all 25 disciplines for which data were available.

Words more likely to be found in reviews of women, meanwhile, included “bossy,” “nurturing” and “strict.” The same was true of fashion-related words, such as “frumpy” and “stylish.”

Still, some have criticized the economics forum in the past for allowing targeted discussions of women. One such critic is Melissa S. Kearney, a professor of economics at the University of Maryland at College Park, who thanked Wu on Twitter this week for her work.

Economics Job Market Rumors reposted moderation guidelines after numerous on-site references to Wu's article -- some of them critical, and some of them laudatory. 

Via email, an unnamed administrator for the forum said Wu’s paper “shows that we can do a better job in the economics profession, and [the forum] will definitely do its part to assist with this.”

The site has a “very strong policy against sexism, racism and homophobia,” which is enforced by a team of male and female moderators and a trained bot, the administrator said. "We have improved things dramatically over recent years, with moderation becoming stronger and user attitudes starting to move in the right direction.”

At the same time, users of the forum “recognize the enormous value that the site provides to people in the economics profession,” the administrator said. In addition to offering information on the job market and current research, the administrator added, the site has helped to strengthen scientific rigor in economics by identifying errors and highlighting conflicts of interests in major papers. 

Hengel said Econjobrumors.com “is not without merit,” in that, if “you’re willing to wade through a lot of junk, you will find many thoughtful posts — some of which were really helpful to me when I was on the job market."

Unfortunately, she said, it “likely comes at a cost, by placing a disproportionate burden on women and other minorities that I’m glad is finally being talked about.”

Jeffrey R. Brown, the Josef and Margot Lakonishok Professor of Business and dean of the College of Business at the University of Illinois at Urbana-Champaign, wrote on his blog that it would be easy to dismiss Wu's findings "on 'intellectual' grounds," since there is little doubt that there is "sample selection bias in favor of trolls and malcontents." 

That's beside the point, however, he said, adding, "This should not be happening at all, and senior, tenured, visible economists – especially we men who numerically dominate the profession – have a shared responsibility to change it."