Gamified inoculation interventions do not improve discrimination between true and fake news: reanalyzing existing research with receiver operating characteristic analysis
Modirrousta-Galian and Philip Higham reanalysed the evidence that gamified inoculation games such as Bad News and Go Viral teach people to spot fake news, in the Journal of Experimental Psychology: General. Their contribution is methodological rather than a new experiment. The existing studies had participants rate the reliability or manipulativeness of true and fake news items before and after playing, usually against a control group playing Tetris or doing nothing, and compared mean ratings. That design cannot tell two different things apart. Discrimination means the gap between how a person rates true and false items widens, which is the advertised benefit. Response bias means they rate everything lower, which is not. Both show up as a fall in the average rating of fake items. Reanalysing those studies with receiver operating characteristic analysis, the standard tool for separating the two, the paper reports no improvement in discrimination, and instead more sceptical responding to true and fake items alike. The public record for the paper describes five earlier studies as the material reanalysed. This is a null result on the claim the deployments were justified by, and it is directly at odds with the more optimistic reading those original evaluations invited. The disagreement is about how the data were analysed, and it is not settled by this paper alone. It is a reanalysis of other people's data rather than a new test, and the follow up by Seabrooke, also on this site, did not reproduce the shift in bias that this paper reports.
Modirrousta-Galian and Philip Higham reanalysed the evidence that gamified inoculation games such as Bad News and Go Viral teach people to spot fake news, in the Journal of Experimental Psychology: General. Their contribution is methodological rather than a new experiment. The existing studies had participants rate the reliability or manipulativeness of true and fake news items before and after playing, usually against a control group playing Tetris or doing nothing, and compared mean ratings. That design cannot tell two different things apart. Discrimination means the gap between how a person rates true and false items widens, which is the advertised benefit. Response bias means they rate everything lower, which is not. Both show up as a fall in the average rating of fake items. Reanalysing those studies with receiver operating characteristic analysis, the standard tool for separating the two, the paper reports no improvement in discrimination, and instead more sceptical responding to true and fake items alike. The public record for the paper describes five earlier studies as the material reanalysed. This is a null result on the claim the deployments were justified by, and it is directly at odds with the more optimistic reading those original evaluations invited. The disagreement is about how the data were analysed, and it is not settled by this paper alone. It is a reanalysis of other people's data rather than a new test, and the follow up by Seabrooke, also on this site, did not reproduce the shift in bias that this paper reports.
Written from the citation as recorded when this source was checked, and the public record for the paper.
What our sources record
Reproduced as it was written when this source was checked, figures and all.
Describes the prior pre-post designs, in which participants rated reliability or manipulativeness of true and fake items before and after playing Bad News or Go Viral, usually with a control group playing Tetris or doing nothing, and notes those studies did not separate discrimination from response bias. Verified at the published record 2026-10-05. doi:10.1037/xge0001395
Where we use it
- Teaching People to Doubt Everything, Why We Look
Reliability of this reference
No persistent link has been resolved for this source yet.
The source
No link available. The citation as the article records it:
Modirrousta-Galian, A., & Higham, P. A. (2023). 'Gamified inoculation interventions do not improve discrimination between true and fake news: reanalyzing existing research with receiver operating characteristic analysis.' Journal of Experimental Psychology: General, 152(9), 2411-2437. Describes the prior pre-post designs, in which participants rated reliability or manipulativeness of true and fake items before and after playing Bad News or Go Viral, usually with a control group playing Tetris or doing nothing, and notes those studies did not separate discrimination from response bias. Verified at the published record 2026-10-05. doi:10.1037/xge0001395