An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports
Scheel, Schijen and Lakens compared what gets published when the decision to publish is made before the results are known. Registered Reports are reviewed and accepted on the basis of the question and the method, so a study cannot be rejected for finding nothing. Comparing the first hypothesis tested in seventy-one published Registered Reports with a random sample of a hundred and fifty-two ordinary hypothesis-testing papers in psychology, they found that the standard literature reported a positive result almost every time, in around ninety-six per cent of papers, while the Registered Reports reported one in fewer than half. That gap is the clearest available measure of how much the published record is shaped by what happens after the results arrive rather than by what is true. The authors discuss the possible explanations and conclude that reduced publication bias and fewer false positives are the plausible ones. This vault cites it whenever the reliability of a published literature is at issue.
Scheel, Schijen and Lakens compared what gets published when the decision to publish is made before the results are known. Registered Reports are reviewed and accepted on the basis of the question and the method, so a study cannot be rejected for finding nothing. Comparing the first hypothesis tested in seventy-one published Registered Reports with a random sample of a hundred and fifty-two ordinary hypothesis-testing papers in psychology, they found that the standard literature reported a positive result almost every time, in around ninety-six per cent of papers, while the Registered Reports reported one in fewer than half. That gap is the clearest available measure of how much the published record is shaped by what happens after the results arrive rather than by what is true. The authors discuss the possible explanations and conclude that reduced publication bias and fewer false positives are the plausible ones. This vault cites it whenever the reliability of a published literature is at issue.
Written from the abstract, OpenAlex.
What our sources record
Reproduced as it was written when this source was checked, figures and all.
Scheel, A. M., Schijen, M. R. M. J., & Lakens, D. (2021). "An excess of positive results: comparing the standard psychology literature with registered reports." AMPPS, 4(2). 96.05% of 152 standard articles concluded their first hypothesis was supported, against 43.66% of all 71 published Registered Reports.
Where we use it
- What Forty-One Participants Buys, Why We Look
Reliability of this reference
The link below was matched against the publisher's record on title, first author and year, and all three agree.
The source
https://doi.org/10.1177/25152459211007467
DOI: 10.1177/25152459211007467