Confirmatory Hypothesis Testing

Confirmatory Hypothesis Testing

Reviewed by Faisal Zaidi

Treating an advertisement as a tentative hypothesis and then searching the product in a way that tends to confirm it. The claim, from Stephen Hoch and Young-Won Ha in 1986, is not that people believe whatever they are told. It is that the advertisement arrives first and decides what the experience will be a test of. Joshua Klayman's correction, published a year later, has to travel with it, because confirmatory search is often a sound way of finding things out rather than an error.

What it is

Hoch and Ha's framing is that consumers treat advertisements as tentative hypotheses that can be tested through product experience. In their first experiment, people who saw ambiguous evidence about quality not only judged the product in line with the advertising, their inspection protocols recorded them searching it confirmatorily. The second experiment put the mechanism in encoding: the advertising changed how the physical evidence went in, with some retrieval of advertisement-consistent evidence as a smaller contribution. The commercial significance is in the order of events, because the advertisement is not competing with the experience so much as framing the question the experience answers.

In effect

The condition under which any of this happens is ambiguity of the evidence, and where the evidence was unambiguous the advertising was inert. That makes the practical reach of the mechanism a question about markets rather than about persuasion, which is why Hoch's later observation that quality parity is the norm in many categories does so much work. Hoch and John Deighton's 1989 managerial framework names where a firm can get at this: hypothesising, exposure, encoding and integration, moderated by familiarity with the domain, motivation to learn, and the ambiguity of the information environment. It is a framework and not a study, and should be attributed by name as one.

What it does not say

It does not say that confirmatory search is irrational. Klayman and Ha argued in 1987 that many phenomena labelled confirmation bias are better understood as a general positive test strategy, which under realistic conditions can be a very good heuristic for determining whether a hypothesis is true, though it can also produce systematic errors and inefficiencies. This page therefore cannot be used to imply that a person testing a product against an advertisement is doing something defective. It also carries no numbers: the 1986 paper is closed, the abstract gives no sample size, no stimuli and no effect size, and none may be printed, so the mechanism rests on the authors' description of their own protocols rather than on a published figure.

Sources

  1. Hoch, S. J., & Ha, Y.-W. (1986). "Consumer Learning: Advertising and the Ambiguity of Product Experience." Journal of Consumer Research, 13(2), 221-233. doi:10.1086/209062. Abstract read in full. No sample size, stimuli or effect size available, and none printed.
  2. Klayman, J., & Ha, Y.-W. (1987). "Confirmation, disconfirmation, and information in hypothesis testing." Psychological Review, 94(2), 211-228. doi:10.1037/0033-295X.94.2.211. Abstract read in full; Klayman first author, with the same Young-Won Ha second. A theoretical and analytical paper with no sample.
  3. Hoch, S. J., & Deighton, J. (1989). "Managing What Consumers Learn from Experience." Journal of Marketing, 53(2), 1-20. doi:10.1177/002224298905300201. Abstract read in full. The four-stage framework with three moderators.
  4. Hoch, S. J. (2002). "Product Experience Is Seductive." Journal of Consumer Research, 29(3), 448-454. doi:10.1086/344422. Full text read. The source for quality parity as the norm in many categories, stated as an assertion rather than measured.