Half the Sample

Men and women have a similar chance of contracting COVID-19, but the death rate for men who do get it is more than double that for women. This difference has implications for treatment and public health. And it's just one glaring example of why it’s important to include both male and female subjects in biological research: not every condition or phenomenon affects men and women -- or male and female animals or cells, where sex can be determined -- the same way.

Even so, scientists have historically excluded female research subjects in their studies. In so doing, many cite regularly fluctuating levels of the hormones progesterone and estrogen within female subjects and their potential to skew the data.

Research has since challenged the assumption that female subjects make for wildly variable findings. And the practice of excluding female subjects in research, known as sex exclusion, has been criticized as scientific sexism. If an effect on male subjects is worth studying, these critics argue, it’s worth studying among female subjects, too.

Popular interest in "data bias" against women today is high. But one of the biggest alarm bells on sex exclusion sounded more 10 years ago, in the form of a study of biological research. Just 28 percent of some 800 studies included female subjects, according to that analysis.

Sex Matters

Have things changed in terms of sex exclusion in biological research over the last decade? Yes and no, but definitely not enough, according to a new study of more than 700 journal articles, published in eLife.

Some 49 percent of studies across nine biological fields included both male and female research subjects in 2019, up significantly from the data from 2009. Yet there was no increase over time in the share of studies to analyze the data concerning female subjects. Most studies just lumped in the males with the females.

Some papers excluded the specific number of male and female subjects studied, denying other scientists that basic information. Just 4 percent of the studies included information on why they’d excluded female subjects.

That’s all despite a 2016 National Institutes of Health directive that scientists consider sex as a biological variable in all their funded preclinical research on vertebrate animals and human cells and tissues.

“In the last 10 years, there has been a major in increase in sex inclusion, but it’s still not where it’s needs to be,” said the new paper’s lead author, Nicole C. Woitowich, associate director of the Women's Health Research Institute and research assistant professor at Northwestern University’s Feinberg School of Medicine.

Especially “troublesome to me as a scientist,” Woitowich continued, is that “a little under one-third [of studies] did not even report the number of males and females used as subjects.” Replication of these studies -- a basic tenet of science -- becomes impossible “if I don’t have all the information. I’m left guessing.”

The pattern could amount to major health inequities between men and women, since female-specific data aren't being regularly examined in all kinds of research that could have implications for their health, Woitowich said. Women’s health science shouldn’t be an afterthought, or confined to what’s cynically been referred to as “bikini” science -- or reducing women’s health science to being about or related to the reproductive organs.

Woitowich said she was unfortunately uncertain as to how many researchers are even aware that this is a problem.

“Many scientists, I don’t even know if this is on their radar,” she said. One possible solution is that research subject sex discussions be included in any basic research ethics and protocol class for graduate and medical students.

Because so many researchers apparently have a blind spot here, Woitowich said the real change agents may need to be the science “gatekeepers,” including funders, journal editors and peer reviewers. She asked why so few published studies had offered thoughtful considerations of subject sex, despite the NIH policy.

Sex exclusion was a topic of conversation for several years at the NIH before it issued its policy. In a 2014 comment in Nature, for example, NIH leaders Janine A. Clayton and Francis S. Collins said that while about half of participants in NIH-funded clinical research are currently women, “There has not been a corresponding revolution in experimental design and analyses in cell and animal research -- despite multiple calls to action.”

Publications often “continue to neglect sex-based considerations and analyses in preclinical studies” and reviewers, “for the most part, are not attuned to this failure,” Clayton and Francis said. “The over-reliance on male animals and cells in preclinical research obscures key sex differences that could guide clinical studies. And it might be harmful: women experience higher rates of adverse drug reactions than men do.”

Sex has been proven to have implications for dosages of certain drugs and even the science of addiction, among other things, they also said.

Addressing concerns highlighted in Woitowich’s paper and elsewhere, the pair wrote that “inadequate inclusion of female cells and animals in experiments and inadequate analysis of data by sex may well contribute to the troubling rise of irreproducibility in preclinical biomedical research, which the NIH is now actively working to address.”

The Winding Road to Sex Inclusion

Not everyone supports the NIH policy or sex parity in research design, however, for reasons practical, scientific and philosophical.

In 2015, for instance, a group of scholars wrote in Science that the policy’s focus on “nonhypothesis-driven documentation of sex differences in basic laboratory research is more likely to introduce conceptual and empirical problems in research on sex and gender than bring new clarity to differences in men's and women's health outcomes.”

For example, they said, gender and body weight may have more to do with how some drugs affect men and women differently than the basic concept of sex. Women are more likely to take multiple pharmaceuticals at one time than are men, and they tend to be lighter, and neither factor is inextricably linked to sex at birth. Nonhuman research models pose vexing questions of their own, such as whether cell lines can be definitively declared male or female, and how to approach sexual dimorphism and differences in behavior and developmental pathways across common lab animal species.

If the goal is to advance human health, the scholars wrote, “we see a stronger empirical basis for directed funding initiatives in two areas: scientific validation of preclinical models for studying human sex differences, and human studies of the interaction of sex- and gender-related variables in producing health outcomes that vary by sex.”

Sarah S. Richardson, professor of the history of science and women, gender, and sexuality studies at Harvard University and a co-author of that commentary in Science, said the new study perpetuates what she calls “binary sex essentialism.” She instead promotes “sex contextualism,” or the “simple view that the definition of sex and sex-related variables, and whether they are relevant in biological research, depends on the research context.”

Sex contextualism doesn’t deny that sex is an evolved developmental pathway, or that basic male-female comparisons are sometimes apt research designs for studying sex. But there are “many ways of operationalizing sex in biomedical research,” of which male-female comparisons are only one. Where they’re made, these comparisons are limited in how they can be generalized to all “females” and “males,” especially to humans.

Is there any metric by which researchers may assess their progress in attending to sex-related variables, if not by including “both” male and female subjects then? Richardson said yes. It's the "precision with which researchers are specifying whether, how and why they use sex" as a biological variable in a given research program. In other words, what matters isn't whether research designs always incorporate a balance of male and female subjects, but rather whether sex is being thoughtfully considered as a possible variable.

At least according to Woitowich’s paper, progress on this front is lagging.

Why is this? Availability of materials and funding are roadblocks for other scientists working toward sex inclusion, according to a 2019 analysis by Harvard University’s GenderSci Lab. Annika Gompers, a researcher at the lab, interviewed basic science researchers and found that they “repeated again and again that their experiments were in large part constrained by practicalities of various sorts. These practicalities were often used to explain why they don’t or can’t account for sex in their research.”

Whether their studies involved rats, pancreatic cells from dead human donors or discarded fat cells sourced from cosmetic surgeries, the researchers all generally said they take what they can get. The researchers all expected their costs, along with complexities, to increase with incorporating male and female cells. This was particularly concerning to small labs struggling with funding.

“While my participants recognize that considering sex can be important for the generalizability of results, in practice it is often impractical if not impossible to incorporate sex as a variable into biomedical research,” Gompers wrote. “Such a finding is consistent with scholars who have long looked at science as practice and observed how practicalities -- as mundane as the availability of materials -- are often central to the reduction of complexity into ‘doable problems.’”

‘Losing Statistical Power’

Angela Saini, a science journalist and author of Inferior: How Science Got Women Wrong and the New Research That's Rewriting the Story, was also skeptical about mandating sex inclusion.

“While it's valuable to include a broad spectrum of people in studies,” including women, she said via email, “it doesn't necessarily follow that the sex differences will be significant or important. So disaggregating for sex, while useful sometimes, doesn't always matter.”

Beyond that, Saini said “we need to be careful of assuming that men and women have completely different bodies biologically because they don't, and sex difference research has historically been used as a stick to beat women with. It's useful to investigate the differences where they matter, but beyond that it can become a dangerous game of essentialism.”

Woitowich’s co-author on the study is Anneliese Beery, associate professor of psychology, neuroscience and biology at Smith College -- and an author of the 2009 baseline study of sex inclusion in biomedical research.

Addressing some of the criticisms of sex inclusion as a default approach to basic science, Beery said that understanding the effects of sex as well as gender remains “important at multiple levels of analysis.”

While some scientists worry that sex inclusion in preclinical research will require them to, say, double their sample sizes, that’s “not actually necessary,” she said. Modest increases in sample sizes can still capture sex differences, with the help of factorial analysis, for instance. Major observed differences can be followed up on, where they occur.

Of her new study, Beery said that it's "exciting to see increased inclusion of female subjects across so many different fields of biology." No improvement in the percent of studies that considered subject sex when analyzing the data "means we are still missing out on the opportunity to understand when there are sex differences," however, "and losing statistical power when sex differences go unnoticed."