The Research Reproducibility Crisis

Let me ask you a question. What percentage of published research do you think is reproducible? Eighty per cent? Ninety per cent? What if I asked you how much of published, peer-reviewed research you think should be reproducible? Nearly everyone would say one hundred per cent.

Well, I’ve got some bad news: you’re not even close. One prominent report (Begley & Ellis, 2012, Nature) found that, of ffty-three landmark cancer-research studies, just six fndings (11%) were reproducible. Some weren’t even reproducible in the original investigator’s laboratory. These kinds of studies can and have spawned further research and clinical trials, leading researchers, and indeed patients, down paths with dead ends.

This lack of reproducibility in research is known as the Research Reproducibility Crisis. The crisis spans the biological sciences, economics, psychology, and chemistry, among others. It exists, yet most people aren’t aware that it does. As a biology PhD student, I wasn’t even fully aware of the depth of the issue until I attended the Reproducible Research Oxford launch event in January. A Wellcome-funded survey of more than 140,000 people in 140 countries found that 74% of people trusted scientists. Imagine if they all knew what scientists know: that researchers themselves can’t even reproduce their own work. Would people listen when scientists speak the truth if scientists can’t agree on what is “true”? Or be OK with governmental or charitable funding of research? Trust in scientists and scientifc institutions would surely go down. If we can’t efectively manage this crisis the consequences for public trust in science could be dire.

So, what are the underlying causes? “Publish or perish” is a pervading mantra in the common rooms of universities and research institutes worldwide. This idiom illustrates the belief held by researchers young and old that as academics we must publish often, and in high-impact journals, to survive and thrive in our felds. Early-stage career researchers, in particular, can attest to that. It’s easy to see how this pressure could lead to conscious and subconscious failures to adhere to good, rigorous, scientifc practice. Given the volume of research published annually, and the incentives for generating high-impact research (money, publicity, career progression, even career survival), the crisis we now face is hardly surprising.

The unenviable task now before us is that of changing the culture of research and shifting the incentive from getting research published to getting it right.

Left and right Alexandrina Von Mann