The Population to Individual Gap

What it is

An average is a statement about a set. Almost every inferential claim in behavioural science, neuroscience and machine learning is a statement about a set, and almost every commercial use of such a claim is a statement about a person.

The gap is not a technicality and it does not shrink with sample size. A larger study makes the average more precisely estimated. It does not make the average more true of any particular member of the set, because the variance is a property of the population rather than of the measurement.

Two words carry the whole distinction, and emotion recognition supplies them. The 2019 consensus review finds the predicted facial configurations occurring with the predicted emotion above chance, and not sufficiently reliable and specific across contexts, individuals, and cultures to be considered diagnostic displays of any emotional state. Above chance is a population statement. Diagnostic is an individual one. The entire commercial proposition of emotion recognition sits in the space between those two words.

The contrast that makes it sharpest is with a case where the gap can be closed. A diagnostic model in medicine can be wrong about one patient, and known to be wrong, because a biopsy exists to contradict it. For an emotion read off a face there is no biopsy: the review states that there are currently no objective measures that reliably, uniquely and replicably identify an instance of one emotion category against another. A system can therefore be right on average and unfalsifiable in the individual case at the same time, which is a worse epistemic position than being merely inaccurate.

The same structure shows up wherever a model is deployed at scale. A thousand human assessors are biased in a thousand slightly different directions, and somebody rejected by one has a real chance with the next. One model deployed across a thousand sites is biased in one direction, and somebody it misreads is misread everywhere. Scale removes the scatter that used to function as mercy, and that argument does not require the model to be worse on average than a human.

In effect

In research: the largest cross-cultural result is a population result

Alan S. Cowen and colleagues report sixteen expression types recurring in similar contexts across six million videos from 144 countries, with each type's associations to context 70 per cent preserved across twelve world regions. That is a real and large finding about distributions, and the authors present it as one. It licenses nothing about the face in front of a particular camera at a particular moment, and the paper does not claim otherwise.

In everyday life: the variance caveat nobody quotes

A regional brain activation, a personality correlation, a reaction-time difference: each is reported as a group average over a distribution that includes people in whom nothing happens and people in whom the opposite happens. The useful habit on reading any such claim is to ask what share of individuals showed the effect, which is almost never reported.

In commerce: the per-person inference being sold

A retail analytics product that reports a shopper's emotional response to a display is making an individual claim from population evidence. The honest version of the same product reports that a face moved into a configuration, which is a camera question and one cameras are good at. The difference between those two products is not accuracy. It is what the claim is about.

What it does not say

It does not say population findings are useless. They are the right basis for designing systems, setting policy and allocating resources, all of which act on sets.

It does not say individual claims are impossible. It says they need individual evidence: a within-person measurement, a convergent measure, or a criterion that can contradict the inference.

It does not supply a threshold at which an average becomes applicable to a person, because none exists. The question to ask of any per-person claim is what would show it to be wrong in this case, and where nothing would, the claim is not weak evidence but not evidence.

It is also not a research finding in its own right, but a standing methodological caution this publication applies, and the specific figures on this page belong to the papers named below.

---