Structured Interview

Structured Interview

Reviewed by Faisal Zaidi

A structured interview is a hiring interview in which every candidate is asked the same predefined questions, each assessment is scored separately against a rubric, and the scores are combined only afterwards. In Noise, Daniel Kahneman, Olivier Sibony and Cass Sunstein present it as the working form of two ideas, decomposition and independence, and report that it predicts job performance better than the unstructured conversation most employers still prefer. It matters for the halo effect because the structure stops one early impression from setting every later score.

What it is

The setting is the hiring chapter of Noise, "Structure in Hiring". The authors call personnel selection a perfect test case for structuring complex judgments, because no complex judgment task has been the focus of so much field research, and they remind readers that the first issue of the Journal of Applied Psychology in 1917 had already named hiring "the supreme problem".

Their starting point is how poorly unstructured interviews work. The book gives the correlation between an interview rating and later job performance as about .28, with other studies ranging from .20 to .33, and translates that into a chance of roughly 56 to 61 per cent that the candidate rated higher will perform better. Two interviewers who see the same candidate agree only moderately, with correlations of .37 to .44, and even when a panel holds the candidate's behaviour constant, a meta-analysis puts agreement at .74, so two panellists will still disagree about which of two candidates is better about a quarter of the time.

The structured alternative has three principles. The first is decomposition: break the decision into mediating assessments, which focus attention on the important cues, specify what data is needed, and filter out the irrelevant. The second is independence: collect information on each assessment separately, using predefined behavioural questions, recorded answers and a unified rubric that gives examples of average, good and great answers. The third is delayed holistic judgment: intuition is not excluded, but it is held back until the evidence has been assessed.

In effect

The book's main illustration is Google. Its audit of its own interviews found, as the book puts it, no relationship at all between interview scores and later performance, a result the authors call a random mess. The company had at one point run twenty-five interviews per candidate, and a review concluded that interviews beyond the first four added almost no validity. The rule it enforces is that interviewers rate the candidate separately, before comparing notes with one another. Google's four assessments are general cognitive ability, leadership, cultural fit and role-related knowledge, and the authors point out what the list leaves out: a candidate's good looks, smooth talk and exciting hobbies. The hiring committee then reviews the complete file and decides holistically, but the decision is anchored on the average of the four interviewers' scores.

The reported payoff is modest and specific. Structured interviews correlate .44 to .57 with job performance, a chance of 65 to 69 per cent that the higher-rated candidate does better, against 56 to 61 per cent for unstructured interviews. Candidates and interviewers dislike them, the authors note, and they still recommend them. Work sample tests, they say, are among the best predictors, and Google also uses backdoor references from its own employees.

The link to the halo effect is direct. An unstructured interview invites the interviewer to form a global impression in the first minutes and then read every answer through it. The book describes an experiment in which candidates were told to answer closed questions at random, and not one interviewer noticed that the candidates were answering at random; interviewers in the random condition were as likely to say they could infer a lot about the person as those who met truthful candidates. Scoring each dimension separately, before any discussion, takes away the opportunity for an early impression to colour the rest. The authors trace the lineage of the method to the selection procedure Kahneman built for the Israeli army in 1956, which he also describes in Thinking, Fast and Slow.

What it does not say

The accuracy figures are the book's. This site has not rechecked them against the primary meta-analyses, so they are best read as the authors' summary of the field, not as measured results of this site's own. The authors also accept a circularity in their own evidence: the usual measure of job performance is a supervisor's rating, whose validity they have just questioned in the preceding chapter, and they say any analysis of hiring decisions must make that assumption.

Structure is not a guarantee against the halo effect. It reduces the room for one impression to spread, and the book says nothing about eliminating it. Nor does the chapter claim that intuition has no place; the third principle keeps it and delays it. The claim that structured methods are cheaper, because few things cost as much as face time, is the authors' argument and was not tested here.

The chapter was written for organisations choosing among candidates, and it should not be read as advice about judging people in general.

Sources

  1. Kahneman, D., Sibony, O., & Sunstein, C. R. (2021). Noise: A Flaw in Human Judgment. Little, Brown Spark, with the halo effect and performance ratings.
  2. The page numbers and figures above come from the vault's reading notes on the book, and every statistic is reported as the book gives it.
  3. See also the article that uses the concept, "What is the halo effect?".