Brien, S.
[ABSTRACT READ IN FULL at the arXiv record; the body was NOT READ. Author order as listed on the arXiv record: Kaylee Chhua first] Chhua, K., Wen, Z., Hathalia, V., Zhu, K., & O'Brien, S. (2024). 'From Bias to Balance: Detecting Facial Expression Recognition Biases in Large Multimodal Foundation Models.' arXiv:2408.14842v1. They benchmark four large multimodal foundation models on facial emotion detection 'across different racial demographics', report linear-classifier accuracies on CLIP embeddings of 95.9 per cent on RADIATE, 90.3 per cent on Tarr and 99.5 per cent on Chicago Face, and state: 'we identify that Anger is misclassified as Disgust 2.1 times more often in Black Females than White Females.' Their framing sentence is that such systems 'often exhibit higher error rates for individuals with darker skin tones'. THE PRECISE THING TO SAY: it is a preprint, its headline accuracies come from a classifier the authors trained on embeddings rather than from a product, and the systems benchmarked are general-purpose foundation models rather than emotion-recognition products. Its usable contribution is the SHAPE of the error and not its size: one specific confusion, anger read as disgust, at a stated ratio in one intersectional group. The 2.1 figure is a ratio of a misclassification and is NOT an accuracy difference. If the article cannot carry the preprint label and that distinction in the same breath, it should drop this source rather than simplify it.
[ABSTRACT READ IN FULL at the arXiv record; the body was NOT READ. Author order as listed on the arXiv record: Kaylee Chhua first] Chhua, K., Wen, Z., Hathalia, V., Zhu, K., & O'Brien, S. (2024). 'From Bias to Balance: Detecting Facial Expression Recognition Biases in Large Multimodal Foundation Models.' arXiv:2408.14842v1. They benchmark four large multimodal foundation models on facial emotion detection 'across different racial demographics', report linear-classifier accuracies on CLIP embeddings of 95.9 per cent on RADIATE, 90.3 per cent on Tarr and 99.5 per cent on Chicago Face, and state: 'we identify that Anger is misclassified as Disgust 2.1 times more often in Black Females than White Females.' Their framing sentence is that such systems 'often exhibit higher error rates for individuals with darker skin tones'. THE PRECISE THING TO SAY: it is a preprint, its headline accuracies come from a classifier the authors trained on embeddings rather than from a product, and the systems benchmarked are general-purpose foundation models rather than emotion-recognition products. Its usable contribution is the SHAPE of the error and not its size: one specific confusion, anger read as disgust, at a stated ratio in one intersectional group. The 2.1 figure is a ratio of a misclassification and is NOT an accuracy difference. If the article cannot carry the preprint label and that distinction in the same breath, it should drop this source rather than simplify it.
Written from the aBSTRACT READ IN FULL at the arXiv record; the body was NOT READ. Author order as listed on the arXiv record: Kaylee Chhua first.
What our sources record
Recorded when this source was checked, figures included, with page references and quotation marks removed.
(2024). From Bias to Balance: Detecting Facial Expression Recognition Biases in Large Multimodal Foundation Models. arXiv:2408.14842v1.