Moving Scientific Publishing Toward Social Justice

Science is never neutral. It has a long history of upholding racism, misogyny, ableism, colonialism and other harmful systems of domination. One way these ideas remain a part of science is through academic publications. Too often, top-ranking journals publish articles that are based on or perpetuate exclusionary and harmful ideas, including by falsely defining sex and gender as binary or tying socially driven inequities, such as disparities in income and educational attainment, to genetics.

Last year, much discussion occurred in the scientific community around a now-retracted Nature Communications paper by Bedoor AlShebli, Kinga Makovi and Talal Rahwan that suggested that women scientists were less impactful mentors and mentees. Critiques of this individual paper have already been outlined by scientists and peer reviewers. But it is important to remember that the issues raised during the discussions go beyond that one study.

If the scientific community is to prevent these kinds of problematic studies from being published, structural changes need to be made to the entire scientific publishing process. Such work requires not only that authors and peer reviewers change their research and reviewing norms, but also that editors and journal leadership critically reflect on and change publication standards and practices.

In the following sections, we use the above-mentioned Nature Communications paper as a case study for demonstrating the need for the scientific community to draw on critical social theories in its research and publishing practices. This kind of critical orientation helps the community to identify structural changes that can better align scientific publishing norms with the diversity, equity, inclusion and justice commitments that journals, scientific societies and organizations, and higher education institutions profess. We recommend the following actions to help those of us in science and publishing make the scientific publication process more socially just.

No. 1: Critically examine and challenge existing scholarship. Science can’t truly be rigorous without socially just publication practices. For scientific studies that speak directly to social identities and issues, meeting this expectation requires serious engagement with scholarship in adjacent disciplines relevant to those social topics -- including those in the social sciences and humanities. This kind of meaningful interdisciplinary engagement also asks authors, reviewers and editors to adopt a critical stance when reviewing scholarship, including by challenging how past studies and frameworks may perpetuate stereotypes or harmful power hierarchies.

When reviewing the AlShebli, Makovi and Rahwan article, for example, the editors should have required the authors to more strongly center scholarship on gender biases in academe, gender identity misconceptions and effective mentorship practices. Doing so would have highlighted for stakeholders of the review process that 1) sex is distinct from gender and 2) neither sex nor gender are binary.

Furthermore, the evidence used to evaluate whether or not a certain method or tool (e.g., software like Genderize.io that the co-authors used to infer gender from name) is appropriate for answering a research question shouldn’t be based only on its use in prior publications. We know inferring gender only from a person’s name can misgender people and erase nonbinary, transgender and gender-nonconforming individuals. We also know this method can exclude populations if criteria do not account for different social contexts, including variations in culture and language. Not thinking critically about the tools used in studies doesn’t only risk leading to flawed analysis -- it can also harm marginalized populations.

No. 2: Recognize that research processes always already reflect society’s power dynamics. “Culture and power shape knowledge production by establishing the processes by which understandings are generated and disseminated,” Chandra L. Ford and Collins O. Airhihenbuwa write in “The Public Health Critical Race Methodology: Praxis for Antiracism Research” in Social Science & Medicine. Journals should set evaluation standards that make clear how culture and power are part of the scientific process.

For example, researchers should be expected to identify and challenge how power dynamics infiltrate study design, constructs, methodologies and interpretation. In academe, current systems of power emphasize publications and funding as measures of success. Rather than challenge this norm, the Nature Communications article reinforced it, using the number of publications and their citation indices as proxies for successful mentorship -- despite the well-documented problematic nature of those measures.

Furthermore, the paper primarily defined impactful mentorship through co-authorship, which unnecessarily commodified the mentor-mentee relationship and ignored how: 1) co-authorship does not imply mentorship and 2) effective mentors do far more than help mentees publish papers. Effective mentors may also boost morale, nominate mentees for awards or speaking opportunities and provide strategies for navigating a system not created for women, nonbinary individuals and “persons excluded because of their ethnicity or race” (PEERs). But the article was still published -- even though the authors’ methods did not include those activities, which are disproportionately performed by historically excluded and marginalized researchers. The lack of qualitative assessment of these relationships removes the human connection vital for successful mentorship while also reinforcing problematic academic norms.

No. 3: Acknowledge that data are not neutral. Data may “embody the dominant … assumptions that shape contemporary society” because “all data is manufactured and all analysis is driven by human decisions,” David Gillborn and his co-authors note in the Race Ethnicity and Education article “QuantCrit: Education, Policy, ‘Big Data,’ and Principles for a Critical Race Theory of Statistics.” Researchers, peer reviewers and journal leadership have the ethical responsibility to examine how the data in a study may paint an incomplete picture.

The Nature Communications article, for example, did not account for how the co-authors’ data set spanning “over a century of research” was biased, reflecting the systemic barriers that women scientists have faced. Throughout the 20th century, women scientists were given fewer resources and opportunities, resulting in undervalued and underreported accomplishments. Marginalization has been even worse for PEERs, whose participation and contributions have been further suppressed by white supremacy and colonialism. Yet discussion of intersectionality was absent from the article. Authors, editors and reviewers are responsible for a manuscript appropriately framing data, including identifying how social inequities and power dynamics may manifest themselves in analyses and findings.

No. 4: Think critically about broader impacts and potential harms. Research that advances social justice requires that “researchers use the knowledge obtained through their studies to help disrupt one or more causes of the inequities,” Ford and Airhihenbuwa assert. In the Nature Communications paper we’ve been discussing, the authors concluded that women mentorship pairs were less productive, perpetuating sexist ideas that women have less value than men. This interpretation puts the blame on women rather than the unjust systems that propagate unequal access to resources and produce inhospitable and toxic working environments. Such an interpretation could lead mentors to prioritize working with students who are men, further reducing opportunities for other genders in science.

While the authors of the Nature Communications study noted that several societal factors were “out of the scope of current study,” such factors are crucial for correctly framing research questions and conclusions. Editors and publishers have the power to include into journal review criteria the practice of using societal factors to establish a study’s scope and the discussion of the results.

No. 5: Ensure that editors and reviewers have appropriate expertise. Journal review processes must account for the biases baked into science by re-evaluating how reviewers and editors decide what to publish (or not) and why. For example, how does journal leadership ensure that reviewers and editors assess the potential biases of a study? Do standards exist to ensure reviewer comments are addressed adequately?

To answer these questions, scientific journals need peer reviewers who have expertise in how society intersects with and informs knowledge production. Ensuring this expertise exists in the scientific peer review process means adding experts in education and critical theories (such as critical race theory, decolonial studies, disability studies, queer theory, gender studies and other related areas of scholarship), as well as in science, technology and society studies to the reviewer pool. Editors and reviewers should also represent diverse lived experiences and identities in terms of gender, sexuality, race, ethnicity, disability, nationality and so forth. And finally, we need to revise the publication process when journals and peer review fail.

Moving Forward

We must all remember that “Social injustices … are not located exclusively in the large-scale institutional background, but are found also in individual behaviors and relationships that promote inequality,” as J. W. Hammond writes in the chapter “Toward a Social Justice Historiography for Writing Assessment” in the book Writing Assessment, Social Justice, and the Advancement of Opportunity. This reminder is one that applies not just to the study of educational assessment, but also to the entire scientific process -- from question design to publication. As individuals, we can hold ourselves, our peers and our leadership accountable for creating systemic changes in the scientific publication process. Beyond the practices we mention above, we share five more standards that authors, reviewers and journals can adopt. These include:

No. 6: Examine and acknowledge positionality. Positionality (or reflexivity) statements are one way that authors can explicitly acknowledge and discuss their biases and how they shape findings (see the Gillborn and co-authors piece for an example). These kinds of self-reflection or positionality statements should also become the norm for editors, reviewers and organizations.

No. 7: Foreground methods. Methods form the foundation of the conclusions we draw. The “Materials and Methods” section should always be part of the main text, preferably before the results, so readers can readily contextualize and evaluate findings and interpretations.

No. 8: Define constructs. Review criteria should require authors to sufficiently define and provide supporting references for the constructs that are central to their work, including the underlying assumptions within those constructs.

No. 9: Honor and incorporate critical commentary. Peer review does not stop when the paper is published. Scientists cannot ignore when studies are published despite their lack of rigor -- we have a responsibility to call attention to their flaws. When this happens, journals should address community concerns, such as adding a disclaimer to the top of the publication and references to pieces that flag limitations and critical considerations of the study. Nature Communication’s response to the paper we’re discussing provides one beginning example of this kind of work.

No. 10: Acknowledge science’s long and violent history of exclusion. For journals and the scientific community at large to fulfill their commitments to diversity, equity, inclusion and justice, we must all do our part to interrogate the histories of our fields and acknowledge the harm they have caused.

Breaking the pattern of publishing faulty and harmful research requires looking back through the history of science, grappling with its many harms and using that evidence to build more equitable and just scientific norms. Part of this work includes creating publishing criteria that help move the scientific community beyond the incomplete, incorrect and narrow view that blames historically excluded people for the barriers and limitations they face in science. As scientists, reviewers, editors and leaders of scientific institutions and higher education, we must commit to doing better not only in our publishing but in all of our disciplinary practices if we are, in Ford and Airhihenbuwa’s words, “not only to understand inequities, but to eliminate them."