Wisdom of Crowds

What it is

The effect is statistical before it is psychological. If a number is estimated by many people, and their errors are not all in the same direction, the errors partly cancel when the answers are combined. The combined answer can then be better than most of the individual answers that went into it. Nothing in this requires the crowd to be clever. It requires that the errors differ.

The standard illustration is Francis Galton's report in Nature on 7 March 1907, under the title Vox Populi. At the West of England Fat Stock and Poultry Exhibition, visitors bought numbered cards for sixpence and wrote their name, address and an estimate of the weight of an ox after it had been slaughtered and dressed. After 13 defective or illegible cards were removed there were 787 entries. Galton took the middle value, which he called the middlemost estimate, as the voice of the people, reasoning that every other estimate was condemned as too low or too high by a majority of the voters. He reported it as 1,207 pounds against a dressed weight of 1,198, which is 9 pounds or 0.8 per cent too high, and he judged the result more creditable to the trustworthiness of a democratic judgement than might have been expected.

The record has since been tidied. In a 2014 re-examination in Statistical Science, Kenneth Wallis found that the outcome was 1,197 pounds, that the median entry was 1,208, and that Galton's draft had given the error as 11 pounds, closely 1 per cent, before the published version changed it. The mean of the entries, which Galton gave in a reply to a reader as 1,197, matches the weight, so measured against the corrected outcome the mean had no error at all. None of this weakens the effect. It does show that the founding example is a single contest, with a single animal and a single set of cards, and not a body of replicated results.

In effect

The most direct test of what the effect needs is an experiment by Jan Lorenz, Heiko Rauhut, Frank Schweitzer and Dirk Helbing, published in the Proceedings of the National Academy of Sciences in 2011. It used 144 students at ETH Zurich in twelve sessions of twelve. Each student sat in an isolated cubicle and gave five consecutive estimates to each of six factual questions, among them Swiss population density, the length of the Swiss-Italian border and the 2006 Swiss counts of murders, rapes and assaults.

Between rounds people received different information. In the control condition they saw only their own previous estimate. In the aggregated condition they saw the mean of the previous round's twelve estimates. In the full condition they saw the trajectories of everyone's estimates. Payment depended on accuracy alone, with no reward for agreeing with anyone.

The authors report three effects. Social influence reduced the diversity of the group's estimates without improving collective accuracy. As the estimates converged, the true value moved toward the edge of the range of estimates, so that the crowd became a less useful guide to where the truth lay. And people became more confident even though they were no more accurate.

The pattern gives the effect a practical condition. A crowd's accuracy comes from the independence of its members' errors. Asking people to answer after they have seen what others said is a different procedure from asking them to answer separately, and it should not be expected to behave in the same way. Galton's cards were separate pieces of paper, one estimate per entrant.

Other lines of work show the same dependence from different angles. The hidden-profile paradigm of Garold Stasser and William Titus shows that group discussion tends to circulate what members already share rather than what only some of them know. And the experiment by Abdullah Almaatouq and colleagues in 2021 shows that whether a group beats the most efficient individual depends on how complex the task is.

What it does not say

It does not say that groups are generally smarter than individuals. The effect concerns combining separate estimates of a quantity. It says nothing about what happens when a group discusses, deliberates or works together on a task, and the evidence on those situations is mixed. Groups tend to match the fastest individual and beat the most efficient individual on complex tasks and not on simple ones, and discussion can leave unshared information unshared.

It does not say that any crowd will do. The authors of the Zurich experiment themselves state limits. Their design excluded competition, group pressure and authority, and the feedback was minimal. The effects were strongest for questions of moderate difficulty and for more confident individuals. Collective error declined slightly under social influence in places, partly because of an upward drift in the arithmetic mean, so the result may depend on the question set. Questions with no correct answer, such as election or climate predictions, were not tested. They also report that the arithmetic mean was closer to the truth than a person's first estimate in only 21.3 per cent of cases on the raw data, and in 77.1 per cent after a log transformation, which is why the choice of average matters.

It does not settle questions that have no true value, such as which of two films a particular person will prefer. The Zurich design tested only estimates of quantities that do have one.

And it is not the same as the psychological claim that a group has a general intelligence of its own. That claim, proposed by Anita Woolley and colleagues in 2010, concerns groups working together across tasks, and it is contested. This site uses collective intelligence for both ideas and keeps them apart.

On the book that seeded the article, Colin Fisher's The Collective Edge treats the wisdom of crowds as settled in the body of the text, without Galton or a date, while an endnote sets out boundary conditions. The vault's audit records the endnote as more careful than the sentence it supports. This page's evidence status therefore stands at mixed: the 1907 contest and the 2011 experiment are verified, and the general claim is true only under the conditions above.

---