Project Overview

The Reproducibility Project: Cancer Biology was an 8-year effort to replicate experiments from high-impact cancer biology papers published between 2010 and 2012. The project was a collaboration between the Center of Open Science and Science Exchange with all papers published as part of this project available in a collection at eLife and all replication data, code, and digital materials for the project available in a collection on OSF.

When preparing replications of 193 experiments from 53 papers there were a number of challenges.

2%

experiments with open data

70%

of experiments required asking for key reagents

69%

of experiments needing a key reagent original authors were willing to share

0%

of protocols completely described

32%

of experiments the original authors were not helpful (or unresponsive)

41%

of experiments the original authors were very helpful

Fully designed protocols were submitted to eLife for peer review before conducting the replications - a publishing format called Registered Reports - to ensure that the proposed experiments were of the appropriate rigor and quality. Accepted replication protocols received a commitment in advance to publish the findings regardless of outcome.

Additional challenges were encountered for the experiments that were conducted.

67%

required modifications to complete

41%

of modifications completely implemented

Ultimately, 50 replication experiments from 23 of the original papers were completed, generating data about the replicability of a total of 158 effects. There are many ways to evaluate and characterize replication outcomes, some simplified summaries of the findings include:

  • Replication effect sizes were 85% smaller on average than the original findings
  • 46% of effects replicated successfully on more criteria than they failed
  • Original positive results were half as likely to replicate successfully (40%) than original null results (80%)

Collectively, this evidence suggests opportunities to improve the transparency, sharing, and rigor of preclinical research to advance the pace of discovery.

Discover more about the projectRead more about replicabilityView data & code on OSF

Get Involved

Supporters

Consider supporting more projects like RP:CB and other efforts to increase openness, integrity, and reproducibility of science.

Learn more

Researchers

Improve workflow efficiency with integrated tools, keep track of your research in one place, learn new tools, preregister your work, and more.

Learn more

Institutions

Learn about tools and services for your communities to increase transparency and visibility of research outputs, accelerating discovery and reuse.

Learn more

Journals & Societies

Increase the transparency and openness of research, learn how to signal open practices, increase your conference’s reach, and more.

Learn more

Questions about the project can be directed to contact+rpcb@cos.io.
Share: