Statistical Power

Statistical Power

Reviewed by Faisal Zaidi

Power is the probability that a study would detect an effect if the effect is there. A study with low power that returns a null tells you almost nothing, and a study with low power that returns a significant result is likely to have overestimated the effect it found. Both halves matter, and the second half is the one that built the priming literature.

What it is

Power depends on three things: the size of the effect, the size of the sample, and how much noise the design carries. Raise the sample or lower the noise and power rises.

The first consequence is familiar. An underpowered study that finds nothing has not shown that nothing is there.

The second is less intuitive and did more damage. In an underpowered design, the only results that clear the significance threshold are the ones where noise happened to run in the same direction as the effect. So a small study that succeeds necessarily reports an inflated estimate. Run enough small studies, publish the ones that worked, and the literature fills with effect sizes that are too large by construction, each of them apparently significant.

That is why publication bias and low power are the same problem seen from two angles, and why a literature can look unanimous and be wrong.