[47] Evaluating Replications: 40% Full ≠ 60% Empty
This is a blog post rather than a paper, and it is cited because of what it does with the numbers everyone quotes. Discussion of the large reproducibility project in psychology usually reduces it to a single figure, the proportion of studies that replicated. This analysis partitions the results into three groups rather than two: roughly forty studies that replicated, around thirty that clearly failed, and around thirty where the replication was statistically inconclusive, meaning it was not powerful enough to establish either result. That third group is the one the headline figure hides, and counting it as failure overstates the problem while ignoring it understates it. The blog in question is a well-known outlet for methodological analysis, and it is not peer reviewed.
This is a blog post rather than a paper, and it is cited because of what it does with the numbers everyone quotes. Discussion of the large reproducibility project in psychology usually reduces it to a single figure, the proportion of studies that replicated. This analysis partitions the results into three groups rather than two: roughly forty studies that replicated, around thirty that clearly failed, and around thirty where the replication was statistically inconclusive, meaning it was not powerful enough to establish either result. That third group is the one the headline figure hides, and counting it as failure overstates the problem while ignoring it understates it. The blog in question is a well-known outlet for methodological analysis, and it is not peer reviewed.
Written from the citation as recorded when this source was checked.
What our sources record
Reproduced as it was written when this source was checked, figures and all.
Simonsohn, U. (2016). Data Colada [47], 3 March 2016. Partitions the OSC results into roughly 40 replicated, 30 failed and 30 statistically inconclusive.
Where we use it
- What Forty-One Participants Buys, Why We Look
Reliability of this reference
The link below was matched against the publisher's record on title, first author and year, and all three agree.