The Curse of Knowledge in Instructions
Expert-written instructions fail in a predictable way: the writer cannot unknow the subject, so the steps that feel obvious are the ones left out. The curse of knowledge is well supported in direction and disputed in size, and the evidence points to one repair, which is to test instructions on someone who has never done the task.
Answer to the question
Why do experts write instructions nobody can follow?
Experts cannot easily recall what it was like not to know the subject, so they leave out steps that now feel obvious. In Newton's 1990 tapping study, tappers expected half their songs to be recognised and listeners named 2.5 per cent. The repair is to test instructions on a true beginner, not to reread them.
Key takeaways
- Newton (1990) had 40 tappers predict that listeners would name half of 120 well-known songs; the listeners named three, or 2.5 per cent.
- Camerer, Loewenstein and Weber (1989) found that better-informed people could not set their extra knowledge aside, and that markets cut the error by about half without removing it.
- Nathan and colleagues (2001) found that high school teachers ranked algebra problems for beginners in an order unrelated to how 76 and 171 ninth-graders actually performed.
- Ryskin and Brown-Schmidt (2014) re-ran a false-belief version with 3,074 participants and found an effect less than half the original size, so the direction holds and the size is smaller.
- Minto's page 73 tells the writer that writer and reader are not peers in interpreting the writer's thinking, a method for the same problem that the experiments measure.
A Stanford doctoral student, Elizabeth Newton, once paired up eighty undergraduates. One in each pair tapped out the rhythm of a well-known song and the other tried to name it. Beforehand, the tappers guessed that half of their songs would be recognised. Out of a hundred and twenty songs, listeners named three. Newton's study is the cleanest picture of the curse of knowledge: the tapper hears a tune, and the listener receives knocking.
Anyone who has followed a manual written by its own inventor has been the listener. The working rule is short. Do not judge a set of instructions by rereading them. Hand them to someone who has never done the task, watch where they stall, and count every stall as the writer's error. The rest of this piece is the evidence for why the writer's own reread cannot do that job.
What Camerer, Newton and Hinds each tested
The term comes from Colin Camerer, George Loewenstein and Martin Weber, who used it in a 1989 paper in the Journal of Political Economy. They tested a quiet assumption in economic models, that a better-informed person can reproduce the judgement of a less-informed one. In their experiments better-informed people could not set their extra knowledge aside, even when doing so would have paid. Markets cut the error by about half and did not remove it.
Newton's tapping study, from 1990, put the same error into a task anyone can picture. Her 40 tappers predicted on average that listeners would succeed half the time, with individual guesses running from 10 to 95 per cent. The listeners got 2.5 per cent, because the tappers could not unhear the song.
Pamela Hinds then moved the test to instructions. In a 1999 paper in the Journal of Experimental Psychology: Applied she asked people with different levels of expertise to predict how long novices would need to learn a new mobile phone task. Her abstract reports that the more expert the predictor, the worse the prediction, with novice completion times underestimated, and that the debiasing methods she tested did not remove the bias.
The phone task is close to real use. A learner working from printed steps cannot interrupt the author to ask what a step means, so every omission lands on the reader as lost time or a wrong turn. Newton's listeners were in the same position. They had only the knocks, and no way into the melody in the tapper's head. Camerer and colleagues' markets, Hinds's phones and Newton's songs are three settings for one fault: the person who knows cannot reconstruct the position of the person who does not.
How large the effect really is
The curse of knowledge is well supported in direction and disputed in size. Ryskin and Brown-Schmidt (2014) re-ran a false-belief version of the task, first reported in 2007, with 3,074 participants across seven experiments. The direction held. The effect came out at less than half the original size, a knowledge effect of 0.20 against 0.47 in standardised units.
People do credit others with what they themselves know. A claim that experts mislead novices by a fixed amount would go further than any single study. The 1989 markets, the tapping task and the phone task measure different things in different units, so their sizes cannot be compared with each other, and the replication covers only the false-belief version.
[Why Experts Fall for Scams](https://thewhyweproject.com/articles/why-experts-fall-for-scams) reports a UK regulator's finding that scam victims often know more than average about the subject of the scam, another case where knowing more did not protect. [The Psychology You Learned Didn't Replicate](https://thewhyweproject.com/articles/the-psychology-you-learned-didnt-replicate) sets out how to read a psychology result after the replication audit, which is the position this one is in.
A teacher's ranking of what beginners find hard
The sharpest classroom case of the curse of knowledge is the expert blind spot, reported by Mitchell Nathan and colleagues in a 2001 conference paper. They gave 105 school teachers a set of algebra problems and asked them to rank the problems from easiest to hardest for beginning algebra students. Two samples of ninth-graders, 76 and 171 students who had finished a year of algebra, supplied the real answer. The students solved equations correctly under 30 per cent of the time and word problems over 50 per cent of the time.
High school teachers (39 of them) expected the reverse. They judged the story problems hardest, on the reasoning that pure symbols come most easily, and their ranking bore no relation to how students actually did (a rank correlation of zero). Middle school teachers (30) ranked the problems in close agreement with the students (a rank correlation of 0.73). Knowing the subject well did not help the high school group see it as a beginner does. For an instruction writer, the steps that feel obvious are the ones most likely to be missing.
Warnings do not switch the error off either. In four experiments by Cindy Xiong and colleagues, students read a backstory that made one pattern in a graph stand out to them, then predicted what a viewer who had not read it would notice. They were told to ignore what they knew. They still predicted that the naive viewer would see the pattern the backstory had made salient. The experiments used between 17 and 29 students each, and the effect was weaker when the graph carried no visual annotations. It is a small-sample result, and a consistent one.
Minto's workaround, and the test that follows from it
Barbara Minto, a former McKinsey consultant, built a writing method from editing other people's documents, and set it out in The Pyramid Principle (1987). On page 73 she tells the writer that "you and he are not peers in interpreting your thinking", and she builds the rest of the method around that gap: state the conclusion first, group the support beneath it, and add transitions only where the reader's mind needs pulling back. She does not name the curse of knowledge and cites no experiment for it. Her book is a method resting on consulting practice, not a study, and what it offers here is a procedure that fits the problem the studies measure.
Taken together, the studies of the curse of knowledge show what a writer cannot do, and the repair is inference from them. A writer cannot find the gaps by rereading, because the reread supplies the missing steps from memory. Someone outside the subject can find them. A person who has just learned the task, or who is about to, is closer to the reader than the author will ever be, and the cheapest test is to watch one of them try the steps aloud. [Can a Forcing Function Stop Mistakes](https://thewhyweproject.com/articles/can-a-forcing-function-stop-mistakes) shows what design can do where instructions fail.
A writer cannot find the gaps by rereading, because the reread supplies the missing steps from memory.
Pick a tester who has never done the task. Record where they pause, backtrack or ask a question, and treat the earliest of those as the likeliest missing step. Rewrite from what the tester did, not from what they said they understood.
Then run it again with someone new, since by the end of the first test the tester has learned the task and joined the people who can no longer see the gap in it.
Common questions
Why do experts write instructions nobody can follow?
Experts cannot easily recall what it was like not to know the subject, so they leave out steps that now feel obvious. In Hinds's 1999 study the more expert people were, the worse their predictions of novice completion times, which they underestimated. The curse of knowledge makes the gaps invisible to the author.
What is the curse of knowledge?
It is the difficulty of imagining not knowing something once you know it. Camerer, Loewenstein and Weber (1989) showed that better-informed people could not set their extra knowledge aside when judging what others would know, and markets reduced the error by about half.
Is the curse of knowledge a proven effect?
The direction is well supported and the size is disputed. A replication of a false-belief version of the task by Ryskin and Brown-Schmidt (2014) found an effect less than half the original size, so the bias is real but smaller than first reported.
How do you test whether instructions work?
The studies point towards giving them to someone who has never done the task and watching where they stall, a repair inferred from the evidence and not tested by it. Newton's tappers expected half their songs to be recognised and 2.5 per cent were, which is why a writer's own reread is a poor check.
Sources
Camerer, C., Loewenstein, G., & Weber, M. (1989). 'The curse of knowledge in economic settings: An experimental analysis.' Journal of Political Economy, 97(5), 1232-1254. doi:10.1086/261651
Newton, E. L. (1990). The rocky road from actions to intentions (Doctoral dissertation). Stanford University, Department of Psychology.
Hinds, P. J. (1999). 'The curse of expertise: The effects of expertise and debiasing methods on prediction of novice performance.' Journal of Experimental Psychology: Applied, 5(2), 205-221. doi:10.1037/1076-898X.5.2.205
Nathan, M. J., Koedinger, K. R., & Alibali, M. W. (2001). 'Expert blind spot: When content knowledge eclipses pedagogical content knowledge.' Proceedings of the Third International Conference on Cognitive Science. Beijing: USTC Press.
Xiong, C., van Weelden, L., & Franconeri, S. (2020). 'The curse of knowledge in visual data communication.' IEEE Transactions on Visualization and Computer Graphics, 26(10), 3051-3062. doi:10.1109/TVCG.2019.2917689
Birch, S. A. J., & Bloom, P. (2007). 'The curse of knowledge in reasoning about false beliefs.' Psychological Science, 18(5), 382-386. doi:10.1111/j.1467-9280.2007.01909.x
Ryskin, R. A., & Brown-Schmidt, S. (2014). 'Do adults show a curse of knowledge in false-belief reasoning? A robust estimate of the true effect size.' PLoS ONE, 9(3), e92406. doi:10.1371/journal.pone.0092406