Automation Bias and Human Oversight

What it is

The clearest clinical measurement comes from Thomas Dratsch and colleagues at the University of Cologne, published in Radiology in 2023. Twenty-seven radiologists read 50 mammograms with suggestions from what they were told was an AI system. In 12 of the 40 test cases the suggestion was deliberately wrong. Readers at every level of experience did much worse on those cases. Very experienced radiologists rated 82.3 per cent of mammograms correctly when the suggestion was right and 45.5 per cent when it was wrong. For inexperienced readers the figures were 79.7 and 19.8 per cent.

The pattern shows that experience reduces the pull but does not remove it, and that a wrong suggestion can lead a trained reader to a wrong answer they would probably not have reached alone.

In effect

Human oversight is often treated as a safety net that works by default. The evidence suggests it has to be designed: how the AI's suggestion is shown, whether the clinician forms a view first, and how often the combined performance of person and machine is measured. A related risk is deskilling. Krzysztof Budzyń and colleagues found in 2025 that after AI polyp detection arrived at four Polish endoscopy centres, the detection rate in colonoscopies done without AI fell from 28.4 to 22.4 per cent, in an observational study.

Urs Gasser and Viktor Mayer-Schönberger's Guardrails (2024) argues for keeping people in charge of decisions as a matter of value. That case stands independently of whether oversight improves accuracy, which is the question this research addresses.

What it does not say

Automation bias does not mean people always defer to machines. Research on algorithm aversion shows people can also abandon an accurate algorithm after seeing it err, and research on algorithm appreciation shows lay people sometimes prefer algorithmic advice. Which way people lean depends on the task, their expertise and how the advice is presented.

Nor does it mean AI assistance is harmful overall. In randomised trials of AI-supported breast screening and colonoscopy, detection rose. The concern is more specific: the human check cannot be assumed to catch the machine's mistakes.

---