Sample Size Neglect
An average from four ratings and an average from nine hundred are read as the same kind of fact. It is the most consequential misreading on any review page.
What it is
A number carried to one decimal place looks like a measurement. It reads as precise in the way a thermometer reading is precise, and the precision does persuasive work that the data behind it cannot support.
What is missing from the display is the only thing that would make the number meaningful, which is how many observations produced it.
In effect
de Langhe, Fernbach and Lichtenstein found that when forming quality inferences and purchase intentions, consumers heavily weight the average rating relative to other cues including price and the number of ratings, and fail to moderate that reliance according to whether the sample is sufficient. They describe the resulting confidence as an illusion of validity.
The everyday version is the perfect score. A five out of five almost always rests on very few ratings, which is precisely the condition that makes an average uninformative.
What it does not say
It does not say the ratings are dishonest. The finding concerns honest ratings and what they can carry.
It also does not say review text is uninformative. A specific complaint from somebody who used the product carries information no average can.
Sources
- de Langhe, B., Fernbach, P. M., & Lichtenstein, D. R. (2016). Navigating by the stars. Journal of Consumer Research, 42(6), 817-833. doi:10.1093/jcr/ucv047
- Where this came from
- No book in this library is the source of this concept. It entered the vault from Read the Count Before the Score, built from primary sources rather than from a book.