Decision Structure Versus Decision Assistance

Decision Structure Versus Decision Assistance

Reviewed by Faisal Zaidi

Stéphanie Mertens's three-way taxonomy of choice-architecture interventions, where the finding is the ordering rather than the average. Interventions that change the organisation and structure of the choices, such as defaults and the physical arrangement of options, consistently outperform those that change the description of the options, which in turn outperform those that reinforce behavioural intentions through reminders, commitments and feedback. The ordering is what makes it useful, because most advice about attention and self-improvement is of the third kind, which is the weakest of the three.

What it is

Mertens, Mario Herberz, Ulf Hahnel and Tobias Brosch's 2022 meta-analysis in PNAS covered more than two hundred studies and over four hundred and forty effect sizes across roughly two point one million people, and put the overall effect in the small-to-medium range, a little over four tenths of a standard deviation. Food choices were the most responsive domain, with effects up to two and a half times larger than elsewhere. The authors report a moderate publication bias towards positive results themselves, which is the sentence their critics took up.

In effect

The dissent is severe and gets full weight. Maximilian Maier and colleagues argued in a Letter in the same journal that the moderate publication bias is the real headline, and that once it is properly corrected no evidence for the effectiveness of nudges remains. Reading the original's own sensitivity analysis, they note that the true effect could be as low as eight hundredths of a standard deviation if bias is severe, and that the reported Egger coefficient meets the published criterion for severe. Using robust Bayesian meta-analysis they find an absence of evidence overall, evidence against an effect for the information and assistance categories, and an undecided verdict for structure interventions. Their own qualification travels with it: every category and domain apart from finance shows evidence of heterogeneity, which means some nudges might work even where there is evidence against a mean effect.

What it does not say

It does not say nudging works, and it does not say nudging fails, which is why the status here is contested and both papers go in with their genres named: one is a research article, the other is a Letter. Note the pattern rather than the averages, because that is what survives. The one category left standing as undecided rather than refuted is structure, which is the same category a separate meta-analysis calls effort, and two independent analyses therefore point at friction and away from intention-reinforcement. The record also notes a reply to the original paper, which has not been read here.

Sources

  1. Mertens, S., Herberz, M., Hahnel, U. J. J., & Brosch, T. (2022). "The effectiveness of nudging: A meta-analysis of choice architecture interventions across behavioral domains." PNAS, 119(1), e2107346118. doi:10.1073/pnas.2107346118. Abstract read in full at the publisher; Mertens first author, University of Geneva. More than 200 studies, over 440 effect sizes, 2,148,439 participants.
  2. Maier, M., Bartoš, F., Stanley, T. D., Shanks, D. R., Harris, A. J. L., & Wagenmakers, E.-J. (2022). "No evidence for nudging after adjusting for publication bias." PNAS, 119(31), e2200300119. doi:10.1073/pnas.2200300119. Full text read; Maier first author, University College London. It is a Letter, not a research article, and anything citing it must say so. The PNAS record notes a reply to the original, which was not read.
  3. Luo, Y., Li, A., Soman, D., & Zhao, J. (2023). "A meta-analytic cognitive framework of nudge and sludge." Royal Society Open Science, 10(11), 230053. doi:10.1098/rsos.230053. Open access, abstract read in full; Luo first author. The independent analysis whose effort category converges with structure.