The Active Control

What it is

The requirement is old and unglamorous, and the reason it matters in this literature is a single number. On Brooke Macnamara and Alexander Burgoyne's coding of 63 growth mindset intervention studies, ninety-four per cent had differences between treatment and control beyond the mindset content, for instance additionally teaching learning strategies, or adding role models and inspirational stories. If nineteen trials in twenty vary two things at once, the literature cannot attribute its own effects to the thing it is named after, whatever the pooled estimate turns out to be. Macnamara is at Case Western Reserve University, Burgoyne at Georgia Institute of Technology, and their paper is preregistered, unfunded and declares no conflicts.

The best-designed control in this literature produced the most awkward result. Yue Li and Timothy Bates, both in the Department of Psychology at the University of Edinburgh, built a condition that told children ability cannot be changed while still recognising their work, explicitly in order to unconfound effort from beliefs about intelligence. In their third study, on 211 children aged ten to twelve in Harbin, there was a significant condition effect on the easier post-challenge items, F(2, 204) = 4.161, with the growth condition improving over the limited condition at beta = 0.26, and the active control producing an effect about thirty-five per cent larger still at beta = 0.36. Their reading is narrow and worth quoting: this contradicts the idea that beliefs about ability being fixed are harmful, and at best supports a role for effort.

The largest test in the literature did it properly, which is part of why its result survives scrutiny. Yeager and colleagues, in Nature in 2019, ran two self-administered online sessions of about twenty-five minutes against an active control about brain function that did not address beliefs about intelligence, with no teacher training and with teachers blind to the study's purpose. Median school fidelity was ninety-seven per cent of screens viewed and ninety-six per cent of open responses written, with a mean within-school response rate of ninety-two per cent.

A good design can also be undone by its reporting. The mindset workshop study described in Dweck's Mindset at pages 230 to 231 has a genuinely well-constructed active control: the same number of meetings, more study skills, as much tutor attention, differing only in mindset content, with teachers blind to allocation. It is reported with no sample size, no magnitude and no statistical test, and the phrase "control group" does not appear anywhere in the book. The gap between a good design and an unreportable account of it is the thing this page exists to name.

And there is a failure mode that no design choice can fix. The Education Endowment Foundation's 2019 effectiveness trial, 5,018 pupils in 101 English primary schools with trained teachers delivering an eight-week programme, found no evidence that participating Year 6 pupils made additional progress in literacy or numeracy on national Key Stage 2 tests. The evaluation also found that many teachers in the control schools were already aware of growth mindset theory, because the practice is so widespread. That is contamination of the comparison, and it makes a null weaker evidence of no effect than it looks. An active control is a design; an uncontrolled control is a measurement problem.

In effect

The control that beat the treatment

The research case is Li and Bates's third study, because it inverts the usual logic of a null. Their active control did not merely match the growth condition, it outperformed it, while telling children the opposite of the message under test and simply acknowledging the work they had done. Whatever moved the easier post-challenge items in that sample, the belief about malleability was not the necessary ingredient. The authors are careful about the scope of that, and so is this site: three studies on 624 children in one city, each powered at about eighty-five per cent for a medium effect.

Why "we tried it and things improved" is not evidence

In ordinary life the active-control problem is the reason a school cannot tell whether its new programme worked. A year group receives six sessions with an enthusiastic outside facilitator, new materials and visible attention from adults, and attainment ticks up. Six things changed. The honest conclusion is that the package did something, and the honest next question is which part, which requires giving another year group five of the six things. Almost nobody does this, because it costs the same as the programme and produces a less flattering answer.

The A/B test that varies two things

The commercial version is a redesign shipped as one release: new copy, new layout, new onboarding, new email. Conversion rises four per cent. The team learns that the release worked and nothing about why, which means the next release cannot build on it. The discipline is identical to the educational one, and so is the temptation, because a single combined treatment is cheaper to build and easier to sell internally than a factorial design that would attribute the gain.

What it does not say

It does not say confounded trials are worthless. A treatment-versus-nothing comparison can establish that a package does something, which is a real finding about a package.

It does not say a null from an active control disproves the mechanism. Li and Bates's own framing is that their result at best supports a role for effort, not that mindset beliefs are inert.

It does not resolve contamination. Where a practice is already widespread, as growth mindset is in English schools, even a well-designed control group may have received part of the treatment.

It is not a complete standard on its own. A trial can have a good active control and still be unusable if, as in the trade account of the mindset workshop, it reports no sample size, no effect size and no test.

Who developed it

No single originator, and this page claims none: active controls are standard experimental design practice. The ninety-four per cent coding is Brooke Macnamara and Alexander Burgoyne's. The best-designed control in this literature is Yue Li and Timothy Bates's at the University of Edinburgh. The largest properly controlled trial is first-authored by David Yeager.

---