Ads Blindness

What it is

The starting point of the paper is a problem with experiments. A product team tests a change on a share of users for a few weeks and compares the numbers. Hohnhold, O'Brien and Tang argue that short-term A/B test results do not always predict long-term effects, because users change their behaviour after a product launches. People learn from what they see, and what they learn alters what they do next.

Ads blindness and ads sightedness are the paper's names for two directions of that learning. In the first, users become less likely to click ads. In the second, they become more likely. The authors report that the change is driven by the quality of the ads previously seen or clicked, including the ads' relevance and the quality of the landing pages they led to.

Put plainly, a user who is shown poor ads, or who clicks one and lands on a poor page, learns something about ads on the product. The lesson shows up later as fewer clicks. A user who is shown ads that deliver learns the reverse. The effect belongs to the person's history with the product, not to the single impression, which is why a test that measures one visit or one fortnight can miss it.

The paper builds a method for measuring this long-term learning and applies it to ads on Google Search. It also builds a model that uses short-term metrics to predict long-term outcomes. It reports two applications: a change to the search ads auction that increased the importance of ad quality, and a 50 per cent reduction in ad load on Google's mobile search interface. Its summary line is that user satisfaction is paramount.

In effect

The effect on practice is a change in what a short-term number is allowed to mean. A team that judges a change by this week's clicks cannot see a user learning to look away over the next few months, because the learning enters the count only after the decision has been made. That sentence is our reading of the paper and not its wording.

The paper does not conclude that short-term metrics are useless. Its model takes them as inputs and uses them to predict the long-term outcome, which treats the count as evidence about something else. The distinction is between a number that is the result and a number that is an input to a forecast of the result. Engagement metrics are usually used in the first way. The paper's authors use them in the second.

The finding links to the other evidence on Engagement Metrics. Clicks are the unit in which advertising is sold and judged, and the history James Williams gives is that those units became the default measures of whole websites. A click is a response to one ad on one occasion. Ads blindness is a record of what many such occasions teach.

It also pairs with Good Abandonment. In that case the same count, no click, means success. In this one the same count, fewer clicks, can mean that users have learned the ads are poor. Both show that a movement in a click number does not carry its own explanation.

The authors are Google researchers writing about Google's own ads product. That is not a flaw in the work, but a reader should know the paper reports from inside the business whose revenue it discusses.

What it does not say

It does not say that showing fewer ads always helps a business. The paper reports a 50 per cent reduction in ad load on mobile search as one application of its method, and this page does not report the size or direction of any result from it, because only the abstract-level description was read.

It does not give an effect size for ads blindness, and none is quoted here. It does not say how fast the learning happens or how long it lasts. It does not claim that the pattern holds for other platforms, other kinds of advertising or other countries. It describes Google search ads in the period of the study.

The term is the authors' name for a measured change in click likelihood. It should not be confused with a general claim that people stop seeing ads, and it is not a claim about attention or perception. It is a statement about clicking behaviour over time.

The vault has not traced whether the term appeared in earlier work, so the originator is credited to the 2015 paper with medium confidence.

---