The Dunning-Kruger Effect

What it is

The original paper, "Unskilled and unaware of it", reports four studies of Cornell undergraduates, in humour, logical reasoning, grammar and a second logic test. In the logical reasoning study, 45 students sat a 20-item test drawn from an LSAT preparation guide. Those in the bottom quarter scored at about the 12th percentile and placed their test performance at about the 62nd. The abstract states the same contrast in one sentence, and it is the figure most often repeated.

The authors' explanation is metacognitive. Metacognition is the ability to tell a good answer from a bad one. If that ability depends on the same knowledge that produces good answers, then people with little of the knowledge have little means of noticing the shortfall. The fourth study was designed to test this. Of 140 students taking a ten-item logic test, a randomly chosen half read a short training packet on logical reasoning and the other half did an unrelated task. Trained low scorers judged their own performance more accurately and were better at telling which of their answers were right. Untrained low scorers did not improve.

The grammar study had a second phase, four to six weeks later, in which the top and bottom scorers graded five peers' tests and rated themselves again. Top scorers raised their estimates, which the authors read as a false-consensus effect, while bottom scorers did not significantly revise theirs.

Adrian Furnham, in the chapter on the effect in The New Psychology (2024), keeps the reading narrow. It is unskilfulness in a particular domain, not low intelligence, and everyone is unskilled at most things. He also notes that the concept has attracted a great deal of critical appraisal.

In effect

The criticism arrived in three forms, and they pull in different directions.

The first is statistical. Joachim Krueger and Ross Mueller proposed in 2002 that regression to the mean and the better-than-average heuristic could produce the pattern. Self-estimates are noisy, people on average rate themselves above average, and any group chosen for an extreme score will sit closer to the middle on a second measurement. Gilles Gignac and Marcin Zajenkowski tested this directly in 2020 with 929 adults, who rated their own intelligence and then sat the Advanced Progressive Matrices. People rated themselves far above the measured result, about 124 against about 102 on an IQ-like scale. Sorted into quarters, the data gave the familiar picture, with the gap shrinking from about 33 points in the bottom quarter to about 13 in the top. Two tests that do not depend on grouping, one for whether errors changed with ability and one for curvature, found nothing. A simulation produced the quartile pattern from data in which none had been built in. They concluded the effect is mostly, rather than entirely, an artefact.

The second is about difficulty. Katherine Burson, Larrick and Klayman ran 12 tasks across three studies in 2006. On moderately difficult tasks the best and worst performers differed little in the accuracy of their judgements, and on harder tasks the best performers were less accurate than the worst. Their account was that judges at every skill level make similar errors and that perceived difficulty moves everyone's estimate together.

The third is a pair of replies. A rational model of the effect, published in Nature Human Behaviour in 2021, showed that an observer with identical insight at every skill level can still produce a group-level pattern by pulling surprising results towards a prior expectation. In a commentary on it, Matan Mazor and Stephen Fleming report that a second model, in which poor performance goes with noisier self-estimates, fitted the online data better, so shrinkage alone does not account for the pattern. They also suggest the effect may be the signature of a single burden rather than a double one. In 2023, Curtis Dunkel, Joseph Nedelec and Dimitri van der Linden repeated the 2020 analyses on a nationally representative American sample of 13,977 people. They noted that more than half of the earlier sample were undergraduates, and found a statistically significant but small effect, with the curved term adding about 1 per cent to the variation explained. They judged it premature to call the effect mostly an artefact.

What it does not say

It does not say that the least competent people are the most confident. The 1999 data are about people who scored in the bottom quarter on a particular test placing themselves too high on that test, and the later work shows the same gap appearing in other forms of analysis.

It does not say that incompetence causes confidence. The 1999 training study shows that teaching a skill improved calibration, which is a narrower claim, and the statistical accounts predict the pattern with no causal story at all.

It does not describe a kind of person. The effect is tied to a domain, and a person who is unskilled at logic may be expert at something else. In the Burson study the best performers judged less accurately than the worst on the harder tasks, so error in judging one's standing is not confined to the low end.

It is not settled in either direction. The strong artefact claim was itself challenged by a replication with a larger and more representative sample. The training experiment is the piece the statistical objections reach least directly, because trained and untrained low scorers were selected on the same test and regression to the mean applies equally to both. That reading is this publication's, and one experiment with undergraduates is a narrow base for a mechanism.

The dispute over the 2020 artefact test did not stop there. In 2023 Avram Hiller argued in a published comment that its equal-error result probably came from how the comparative self-ratings were put on a linear scale, and Gignac and Zajenkowski replied that re-running the analysis with the transformation he suggested again found no support for the effect. Separately, a 2021 rational model by Jansen, Rafferty and Griffiths showed that prior beliefs about ability, or a link between performance and skill at judging each answer, could each produce the pattern, and their large replications favoured the second: low performers being less able to tell whether they were correct, in grammar and logical reasoning. The two lines of work disagree about how much of the pattern is psychology, and neither supports the popular claim that incompetence causes confidence.

---