Review of Noise: A Flaw in Human Judgment
[FULL TEXT READ AT SOURCE, all three pages, at the authors' own deposited copy at sites.stat.columbia.edu/gelman/research/published/noise_review.pdf, located through Gelman's published-papers index. THIS CLOSES THE FILE'S BIGGEST GAP: the first pass could not retrieve it and recorded that honestly] Sood, G., & Gelman, A. (2024). 'Review of Noise: A Flaw in Human Judgment.' CHANCE, 37(3), 70-72. doi:10.1080/09332480.2024.2416879. FIRST, INDEPENDENT CORROBORATION OF OUR HANDLING OF THE KEYSTONE FIGURE, verbatim: 'Take insurance underwriting, for instance. Given the same data (realistic but made-up information about cases), the median percentage difference between quotes for any pair of underwriters is a stunningly large 55% (so for half of the cases, it is worse than 55%), a difference about five times as large as expected by the executives asked about this scenario in a survey.' Two independent statisticians print the median gloss in a parenthesis and the five-times comparison in the same sentence, which is exactly the vault's handling rule. NOTE WHAT THEY DO NOT DO: they do not question the figure's provenance at all, so on that point the article is making an argument these reviewers did not make. THEIR CONSUMER LINE, the shortest useful sentence any source in this file offers a reader: 'If you are a customer, your optimal strategy is to get multiple quotes.' Their two proposed explanations for professional ignorance of the disagreement are offered as conjecture and must be attributed as such: that people may 'anchor' on a colleague's number when they see one, and that 'colleagues plausibly read to agree, asking, "Could this make sense?" rather than, "Does this make sense?"' The anchoring half belongs to the first-price article and should be left there. THEIR CRITICISM: 'Ironically, Noise makes some errors it warns about. Here are two patterns we noticed, that follow
[FULL TEXT READ AT SOURCE, all three pages, at the authors' own deposited copy at sites.stat.columbia.edu/gelman/research/published/noise_review.pdf, located through Gelman's published-papers index. THIS CLOSES THE FILE'S BIGGEST GAP: the first pass could not retrieve it and recorded that honestly] Sood, G., & Gelman, A. (2024). 'Review of Noise: A Flaw in Human Judgment.' CHANCE, 37(3), 70-72. doi:10.1080/09332480.2024.2416879. FIRST, INDEPENDENT CORROBORATION OF OUR HANDLING OF THE KEYSTONE FIGURE, verbatim: 'Take insurance underwriting, for instance. Given the same data (realistic but made-up information about cases), the median percentage difference between quotes for any pair of underwriters is a stunningly large 55% (so for half of the cases, it is worse than 55%), a difference about five times as large as expected by the executives asked about this scenario in a survey.' Two independent statisticians print the median gloss in a parenthesis and the five-times comparison in the same sentence, which is exactly the vault's handling rule. NOTE WHAT THEY DO NOT DO: they do not question the figure's provenance at all, so on that point the article is making an argument these reviewers did not make. THEIR CONSUMER LINE, the shortest useful sentence any source in this file offers a reader: 'If you are a customer, your optimal strategy is to get multiple quotes.' Their two proposed explanations for professional ignorance of the disagreement are offered as conjecture and must be attributed as such: that people may 'anchor' on a colleague's number when they see one, and that 'colleagues plausibly read to agree, asking, "Could this make sense?" rather than, "Does this make sense?"' The anchoring half belongs to the first-price article and should be left there. THEIR CRITICISM: 'Ironically, Noise makes some errors it warns about. Here are two patterns we noticed, that follow
Written from the fULL TEXT READ AT SOURCE, all three pages, at the authors' own deposited copy at sites.stat.columbia.edu/gelman/research/published/noise_review.pdf, located through Gelman's published-papers index. THIS CLOSES THE FILE'S BIGGEST GAP: the first pass could not retrieve it and recorded that honestly.
What our sources record
Recorded when this source was checked, figures included, with page references and quotation marks removed.
Sood, G., & Gelman, A. (2024). Review of Noise: A Flaw in Human Judgment. CHANCE, 37(3), 70-72. doi:10.1080/09332480.2024.2416879