Pivotal Year for Preprints

When medRxiv, a site that hosts unpublished research manuscripts -- called preprints -- in the medical sciences launched in June 2019, things got off to a slow start.

“It may take off slowly,” John Inglis, co-founder of the server and executive director of the Cold Spring Harbor Laboratory Press, told Science at the time.

For the first few months of its existence, that was true. “Medicine was a more conservative discipline,” Inglis, who is also a co-founder of a preprint server for biology called bioRxiv, said in an interview. “We were not expecting an enormous and immediate uptake of preprints, because we needed to create trust, we needed to create familiarity.”

The pandemic changed things.

“In January 2020 we got 240 manuscripts,” Inglis said. “In May of 2020 we got 2,400.”

A huge proportion of those were pandemic-related, with an analysis by Nature suggesting about two-thirds. Between 17 and 30 percent of all research papers about COVID-19 have been preprints, that same analysis showed. Site usage at medRxiv went from 64,000 page views per month in late 2019 to more than 10 million views at the height of the pandemic.

The year 2020 was a banner year for preprints. Submissions to journals also increased during the pandemic, but as the public grew hungry for new information about the virus, preprints captured public attention like never before.

Preprint servers such as medRxiv and bioRxiv have basic guardrails against unethical or dangerous research, but they generally steer clear of evaluating the quality of scholarship. At medRxiv, which was founded by the Cold Spring Harbor Laboratory, Yale University and The BMJ, submissions are looked over by a team of volunteer “affiliates” in the medical sciences, and the site has a rejection rate of about 10 to 20 percent, Inglis said. Papers can be thrown out for not having proper ethical reviews or containing too much patient identification, for instance. MedRxiv also rejects papers that have the potential to cause widespread alarm, such as those claiming a substance is carcinogenic, with the thinking that those papers are best disseminated after peer review.

With the pandemic, posting criteria got stricter. The site banned papers that were based purely on computer modeling, such as modeling the disease spread or projecting the value of therapeutics.

“We felt in silico wasn’t enough,” Inglis said. “[The papers were] often done by people who were frustrated by being prevented from doing their normal work, in their normal labs and were trying to make a useful contribution to the understanding of the pandemic.”

Within higher ed, some preprint papers, including those based purely on modeling, have been influential in guiding conversations. In June, two professors in math and economics modeled the spread of COVID-19 in a university setting, concluding that without any intervention, all students on a campus of 20,000 would catch the virus before the end of the semester. Other papers tracked COVID-19 spread at small colleges. Still other preprint servers, such as arXiv, an older site for math and physics research, have chosen to keep hosting that type of research.

Within science, preprints have attracted some controversy. Supporters often argue that sharing research before peer review accelerates the rate at which science can happen. Without the sometimes lengthy lag of peer review, other researchers can build on findings faster.

Sometimes that acceleration is clear. For example, in June of 2020, researchers from Britain posted a paper to medRxiv detailing their research on the effect of corticosteroids on COVID-19, finding that their use reduced deaths in patients hospitalized for COVID-19. Before the paper was posted on medRxiv, the specific corticosteroid in the study was only used on about 30 percent of patients, according to a study published in the American Journal of Respiratory and Critical Care Medicine based on Australian data. After the corticosteroid article was posted, that number went up to 92 percent.

And the speed at which medicine moves can save lives. According to a study that appeared in PLOS Medicine, more than 90 percent of journal articles examining the epidemiology of SARS weren’t published until after the epidemic ended.

Inglis said it’s difficult to measure the speed of scientific advancement, but that’s not the only rationale for preprints.

“Communicating your work in advance of peer review gives you, the author, a number of advantages, particularly in terms of getting a sense of how the community feels about the work you’ve done,” he said, and incorporating that feedback into your work before submitting a final manuscript.

But while there are lauders of the preprint sphere, there are also detractors. Some argue that making research available before peer review has the potential to boost bad scholarship. Some articles posted to medRxiv or bioRxiv have been withdrawn, such as one that suggested that the structure SARS-CoV-2 has “uncanny” similarities to HIV (which some saw as suggesting foul play).

And when news media picks up bad scholarship, it has the potential to poison public discourse.

A study published in Health Communication this year suggested that about 43 percent of news articles covering preprint research did not accurately frame findings as scientifically uncertain or lacking peer review.

Alice Fleerackers, a Ph.D. candidate at Simon Fraser University in British Columbia, who co-authored the study, said she wants the media to keep covering preprints, but to take more care in doing so. (The study was coauthored with Michelle Riedlinger, Laura Moorhead, Rukhsana Ahmed and Juan Pablo Alperin.)

“If the pandemic really showed us anything, it’s that the ability to do science quickly is super important,” Fleerackers said. “We need to just have a little bit more awareness as a society and as informed citizens that no scientific finding is a fact.”

So far, about 25 percent of COVID-related preprints on medRxiv have been published in medical journals. While some journals, such as the Journal of the American Medical Association, tell researchers that posting on a preprint server could influence acceptance, others, such as eLife, will now only publish those that first appeared on a preprint server.

“There is a tremendous enthusiasm for more transparency, more openness,” said Inglis. “I see preprint servers as being part of that move towards openness and transparency.”