Experience Sampling Method

What it is

The design is simple and the discipline is in the timing. A signal arrives, unpredictably or on a schedule, and the person records what is happening at that moment. Two of the most-quoted numbers in popular psychology come out of this method, and both are better understood once the instrument is visible.

In Wood, Quinn and Kashy's 2002 habit studies, the signal was a programmed wristwatch chiming hourly while participants were awake and the record was a pocket booklet, with entries to be made as events occurred rather than reconstructed later. Compliance was enforced by exclusion: nineteen participants in the first study and sixteen in the second were dropped for completing the form wrongly or retrospectively. That is the compliance problem stated as a number rather than waved at.

The same paper contains the clearest demonstration in either literature that the prompt is part of the instrument. Study 1 asked for the single behaviour at each chime and produced a habit share of thirty-five per cent. Study 2 asked for all behaviours at each chime and produced forty-three. The form changed and the estimate moved eight points, on the same campus with the same criterion. Anybody reading a prevalence figure from experience sampling should ask what the prompt asked for before asking what the number means.

In flow research the instrument's role is larger still, because it supplied the operational definition. In the Chicago programme, an observation counted as flow when the person marked both perceived challenge and perceived skill above their own average for the week, on two ten-point scales. No flow question was asked. Csikszentmihalyi sets this out at page 158 of Flow and calls the criterion rather liberal on the same page, adding that a strict criterion might qualify fewer than one per cent of responses. Sami Abuhamdeh, a flow researcher and a former co-author of his, reports in 2020 that the quadrant criterion often designates twenty-five per cent or more of all daily experience as flow.

The method's own literature records four limitations, none of which Flow concedes. There is a self-report ceiling, because the person is still reporting on themselves. There is selection bias towards compliant volunteers. Compliance is incomplete, commonly around sixty-five to eighty-five per cent for studies of four to seven days at five to seven signals a day. And there is reactivity.

The fifth limitation is the one worth the page. The signal is an interruption, and in flow research the construct under study is uninterrupted absorption, so the unanswered signals are plausibly concentrated in the moments of deepest absorption. The missing data is the data the construct most needs. The 1987 validity and reliability paper by Csikszentmihalyi and Larson exists; it was not read for this page and no finding is attributed to it.

In effect

The eight-point gap between two forms

The research case that most changes how a reader should treat these figures is the 2002 habit pair, because it holds everything constant except the question. Thirty-five per cent and forty-three per cent are the same construct, the same criterion and the same population, measured with two different prompts. The authors say so themselves, explaining that listing several behaviours encouraged participants to include more routine activity. The lesson generalises past habit: a prevalence estimate from experience sampling is partly a property of the form.

A wristwatch chiming while you are halfway through something

In ordinary use the method is intrusive by design, and the intrusion is the point: it catches the moment rather than the memory of it. The cost is visible to anybody who has been beeped while concentrating. Either you stop and record, in which case the state you were in has ended, or you ignore the signal and record nothing, in which case the state is absent from the dataset. For a construct like absorption the second outcome is systematically more likely in exactly the cases the researcher wants.

The dashboard that samples and the survey that remembers

Product organisations run a version of this split without naming it. In-app pulse prompts are experience sampling, cheap and continuous; quarterly satisfaction surveys are retrospective recall, sparse and reconstructive. The two produce different pictures of the same product, and the difference is not noise. Daniel Kahneman's distinction between the experiencing and the remembering self is the same distinction in another register, and a team that only runs the quarterly survey is measuring memory and calling it experience.

What it does not say

It does not remove self-report. The person reporting is still the person being measured, and the method's advantage is over memory rather than over introspection.

It does not deliver a representative sample of a life. A week of signals from volunteers who completed the protocol is a week of signals from volunteers who completed the protocol.

It does not supply a definition. In flow research the method was used to generate one, by treating two above-average ratings as flow, and that criterion is the arbitrary part rather than an output of the measurement.

It does not sample what it interrupts. Nobody has measured how much of the missing data is absorption, so this limitation is a reason for caution rather than a quantity, and the page states it as that.

Who developed it

Mihaly Csikszentmihalyi, born 1934 in Fiume, now Rijeka, took his doctorate at Chicago, joined the Chicago faculty in 1971 and moved to Claremont Graduate University in 1999; he died in Claremont on 20 October 2021. He built the method with Reed Larson and Suzanne Prescott, and the credit is properly shared. Its best-known use outside his own programme is Wendy Wood's habit research at Texas A&M and the University of Southern California.

---