Investigating the replicability of preclinical cancer biology
Errington and colleagues report the Reproducibility Project in cancer biology, which repeated fifty experiments from twenty-three high-profile papers, covering a hundred and fifty-eight separate effects. The results are sobering in two distinct ways. Where the original paper reported a positive effect, the repeated experiment produced an effect around a fifth of the original size at the median, and nine in ten replications came out smaller than the original. And when success was judged by several binary criteria, fewer than half of the positive effects replicated successfully, while null results held up far better. The project also documents how hard the exercise was: many original papers did not report enough detail to repeat the experiment, and the authors had to negotiate protocols. Their conclusion is appropriately careful, that one failed replication does not disprove a finding any more than one success confirms it. This vault cites it whenever the replication problem is described as confined to psychology.
Errington and colleagues report the Reproducibility Project in cancer biology, which repeated fifty experiments from twenty-three high-profile papers, covering a hundred and fifty-eight separate effects. The results are sobering in two distinct ways. Where the original paper reported a positive effect, the repeated experiment produced an effect around a fifth of the original size at the median, and nine in ten replications came out smaller than the original. And when success was judged by several binary criteria, fewer than half of the positive effects replicated successfully, while null results held up far better. The project also documents how hard the exercise was: many original papers did not report enough detail to repeat the experiment, and the authors had to negotiate protocols. Their conclusion is appropriately careful, that one failed replication does not disprove a finding any more than one success confirms it. This vault cites it whenever the replication problem is described as confined to psychology.
Written from the abstract, OpenAlex.
What our sources record
Reproduced as it was written when this source was checked, figures and all.
eLife, 10:e71601. 46% (51/112) replicated, median replication effect 85% smaller, key statistics unobtainable for 68% of experiments.
Where we use it
- What Forty-One Participants Buys, Why We Look
Reliability of this reference
The link below was matched against the publisher's record on title, first author and year, and all three agree.
The source
https://doi.org/10.7554/eLife.71601
DOI: 10.7554/eLife.71601