Source Calibration

What it is

Listeners usually have two cues about a source. One is how confident the source sounds. The other is whether the source turns out to be accurate. Earlier findings showed that each cue helps: confident witnesses are judged more credible than unconfident ones, and accurate witnesses more credible than inaccurate ones. The question the paper asked was whether those two effects operate independently.

They do not. Across two experiments, errors in testimony damaged the overall credibility of witnesses who had been confident about the erroneous testimony more than that of witnesses who had not been confident about it. After making an error, a less confident witness could even appear more credible than a more confident one.

The pattern is hard to explain if confidence and accuracy are simply added together. It is easier to explain if listeners are asking a different question, namely how well the source's confidence tracks its accuracy. A confident error shows poor calibration. An error voiced with doubt shows calibration that is intact. That is the interpretation the authors offer, and the concept takes its name from it.

In effect

For anyone listening to a presentation, the idea shifts attention from how certain a speaker sounds to what the speaker's certainty has been worth. A source that separates what it knows from what it is assuming gives the listener more to work with than one that sounds equally sure throughout, because the listener can see where the confidence is meant to be relied on.

The article on choosing a branding agency applies the idea as a question put to a pitching team: where are you least sure about our brief, and what would you expect to revise after the first round of feedback from real customers? The question invites the team to show its calibration directly. The application is ours, and the experiments themselves tested witnesses.

The concept pairs with the conditions for trustworthy expert intuition. Kahneman and Klein concluded that felt confidence is not a reliable sign of accuracy, and the witness experiments suggest that listeners, once they have an error in front of them, adjust for exactly that gap.

What it does not say

It does not say that confident sources are less credible. In these experiments confident witnesses were judged more credible than unconfident ones, and the extra cost fell on confident errors.

It does not show how listeners judge advisers, specialists or firms. The experiments concerned testimony, and carrying the idea to a pitch is an application and not a result.

It does not give a measure of calibration for buyers to compute. The paper reports how listeners judged witnesses, and it states its reading as an interpretation.

The abstract does not report sample sizes, so this page states none.

---