Pre-Specified Primary Outcome

What it is

Consider a study that measures fifteen things and reports the one that showed a difference. It has run fifteen tests and published one. At the conventional threshold, roughly one in twenty tests on no real effect will clear the bar by chance, so fifteen tests give a reasonable chance of at least one apparently positive result in a world where nothing is happening. Nobody needs to cheat for this to occur. The researcher looks at the data, sees that one measure moved, and finds that measure interesting, which it genuinely is.

Nominating one outcome before the data exists removes the choice. The primary outcome is written into a public registry, with a timestamp, before recruitment. When the results arrive, the study's answer is whatever that measure says, including when it says nothing.

What this converts is not the truth of a result but what the result licenses. An unregistered study reporting a difference on one of fifteen measures has produced a lead worth following. A registered study reporting no difference on the single measure it committed to has produced a test, and a test that came back negative. That is the whole mechanism: pre-specification turns a null from an embarrassment into a finding.

Three implications follow, and all three get dropped in practice.

First, secondary outcomes are not worthless, they are differently weighted. A registered trial that misses on its primary outcome and shows something striking on a secondary one has found something worth a second study. What it has not done is pass its own test. Reporting it as a success is outcome switching, and it is one of the better-documented ways that literatures fill up with interventions nobody can reproduce.

Second, a subgroup result is not a primary outcome unless it was registered as one. "No average effect, but an effect among girls" is a different kind of claim from "an effect among girls", because the subgroups were available to be looked at after the fact and there are usually several of them. This is not a reason to ignore subgroup findings. It is a reason to describe them as what they are.

Third, and most important, a pre-specified null is not a demonstration of absence. The commitment says that the one study designed to be judged on this measure did not find a difference. It does not say the thing does not work. A study can be underpowered, badly timed, run on the wrong population, or measuring the right thing with the wrong instrument, and pre-specification protects against none of that. It protects against one specific failure, which is the retrospective choice of what counted.

The reason the distinction matters so much in politically live territory is that a registered null is the most quotable object in empirical research. It is clean, it is short, and it can be made to mean far more than it does.

In effect

Research: one outcome, named in advance, and no difference

Victoria A. Goodyear and thirteen colleagues, The Lancet Regional Health - Europe, 2025, the study known as SMART Schools, compared 1,227 participants aged 12 to 15 across 30 secondary schools in England: 20 restrictive, where recreational phone use was not permitted, and 10 permissive. The primary outcome was mental wellbeing on the Warwick-Edinburgh scale, registered as ISRCTN77948572 before data collection, and funded by the NIHR Public Health Research Programme.

The result, verbatim: "Mean WEMWBS score was 47 (SD = 9) with no evidence of a difference between groups (adjusted mean difference -0.48, 95% CI -2.05 to 1.06, p = 0.62)." Restrictive schools did show lower phone time during school, an adjusted mean difference of minus 0.67 hours, 95 per cent confidence interval minus 0.92 to minus 0.43, and lower in-school social media time, "but there was no evidence for differences when comparing usage time on weekdays or weekends."

Both halves of that travel or neither does. From the journal's own linked commentary by Weiss and Bonell, read in full: the school response rate was 10 per cent, and the cross-sectional, non-randomised design "limits conclusions about causal effects". So this is a null in an observational study. It is not a demonstration that restriction does nothing. It is a demonstration that the one study with adolescent mental wellbeing as a pre-specified primary outcome did not find a difference between restrictive and permissive schools, which is a narrower and more useful statement.

The quietly consequential detail is the mechanism one: restriction cut phone use inside school and did not cut total daily use.

Research: the contrast case, a null average with a subgroup finding

Sara Abrahamsson, Journal of Human Resources, 2026, peer-reviewed and ahead of print, reports from Norwegian middle schools that "banning smartphones in middle schools has no average effect on education or mental health but masks important gender differences. Bans reduce psychological health care use among girls, lower bullying for both genders, and improve girls' GPAs and academic track placement, especially among those from disadvantaged backgrounds."

Two things make this the right contrast. The headline is a null on the average, with the gender differences underneath it rather than instead of it. And the mental-health outcome counted is take-up of psychological healthcare, which measures use of a service rather than symptoms, and which can move when access, awareness or referral practice moves. This is why the study should never be summarised as "phone bans improved girls' mental health". Give the null average, and name what was actually counted.

Everyday life: the resolution you set in January

Anyone who declared in January that the year would be judged on one thing, and in December counted a different thing that happened to go well, has switched their own primary outcome. The behaviour is entirely ordinary and almost unconscious. What a registered trial has that a person does not is a public record with a date on it, and that is the only difference. The discipline is institutional rather than moral, which is worth knowing because it means the fix is a procedure and not a virtue.

Commerce and product design: the test that was declared on conversion

A team runs an A/B test declared on conversion rate. Conversion does not move. Session length rises. The change ships as a win, and the deck says so.

That is outcome switching, and it is the most common way a product organisation accumulates a year of changes that did not do the thing they were built to do. Every individual decision looks reasonable, because session length genuinely did rise and somebody genuinely does care about it. The organisations that avoid it write the metric into the experiment brief before the traffic is split, which is trial registration with different vocabulary and the same logic.

The newest school-phone evaluation shows the research version of the same discipline. Hunt Allcott and five colleagues, 2026, is pre-registered with a pre-analysis plan on OSF, and it is an NBER working paper that is not yet peer-reviewed, a label that must accompany every citation. Its own reporting carries the hedges intact: "we find suggestive evidence that student subjective well-being falls" in the first year, while "the estimated effects on well-being appear to recover and become positive in later years". The abstract says later years and not a specific number of years, so no year count is printed here.

What it does not say

Pre-specification does not make a study good. A registered trial can be too small, too short, run on the wrong people, or built on a measure that does not capture what anyone cares about, and registration improves none of those. It is one guard against one failure mode.

It does not make unregistered findings false. Most of what is known in psychology was found before registration was normal, and a great deal of it holds.

It does not convert an observational design into a causal one. SMART Schools is cross-sectional, and its own authors and the journal's commentary say so. Its null constrains what can be claimed for school restriction; it does not establish that restriction has no effect, and a randomised trial could still find one.

And nothing on this page licenses advice to a parent about a particular child. The evidence base discussed here is about school-level policies evaluated at population level, and the conclusion of the one pre-specified test available is that no difference in wellbeing was found between restrictive and permissive schools. That is a fact about policy evaluations, and it is not guidance about anybody's household. The reforms most often proposed in this area have never been tested experimentally, a point made most clearly by the author who proposes them: Jonathan Haidt writes in his own endnotes that he knows of no school district that has tested the hypothesis by randomly assigning middle schools to go phone-free, and calls that the most important study he can think of on the question. He then specifies the trial, with whole schools as the unit and measures agreed in advance. That is the study this concept is waiting for, and it is a credit to him that it is on the record in his own words.

Who developed it

No single originator. Pre-specification of a primary outcome is a standing convention of trial registration and reporting rather than one person's proposal, and this publication verified no attribution, so none is given. The vault's note records the origin as unknown.

---