Investigating Bias and Fairness in Facial Expression Recognition

Xu (2020)

[FULL TEXT READ. Author order verified on the arXiv record and in the PDF: TIAN XU FIRST, University of Cambridge; White, Cambridge; Kalkan, Middle East Technical University; Gunes last, Cambridge] Xu, T., White, J., Kalkan, S., & Gunes, H. (2020). 'Investigating Bias and Fairness in Facial Expression Recognition.' arXiv:2007.10075v3. THE BEST-POWERED STUDY OF ACTUAL EXPRESSION RECOGNITION BY DEMOGRAPHIC GROUP, AND IT DOES NOT SAY WHAT PEOPLE EXPECT. Seven-way expression recognition on RAF-DB plus binary smiling on CelebA, with ResNet-18 baseline, attribute-aware and disentangled variants, each with and without augmentation. RAF-DB TEST SET COMPOSITION, Table 1, 2,876 test images of 14,388 total: race Caucasian 77.4 per cent, Asian 15.5 per cent, African-American 7.1 per cent; gender female 56.3 per cent, male 43.7 per cent; age 0 to 3 at 5.5 per cent, 4 to 19 at 16.4 per cent, 20 to 39 at 57.5 per cent, 40 to 69 at 17.4 per cent, 70 and over at 3.2 per cent; expression happy 39.7 per cent, neutral 22.6, sad 13.4, surprise 10.3, anger 5.7, disgust 5.5, fear 2.7. THE WITHIN-LABEL SKEW IS SHARPER THAN THE OVERALL SKEW AND IT IS THE DETAIL THAT MATTERS: of 164 anger images only 10 are African-American and 10 Asian; of 79 fear images, 5 are African-American. CLASS-WISE ACCURACY BY EXPRESSION, baseline without augmentation, mean 65.3 per cent: happy 91.6, neutral 78.7, surprise 75.8, anger 66.5, sad 63.2, disgust 41.1, fear 40.5. MEAN CLASS-WISE ACCURACY BY DEMOGRAPHIC GROUP, baseline, without and with augmentation: male 65.3 and 72.3; female 63.5 and 74.1; CAUCASIAN 65.9 AND 74.7; AFRICAN-AMERICAN 68.1 AND 76.3; ASIAN 60.0 AND 67.8; age 0 to 3 at 63.6 and 80.2; age 4 to 19 at 59.5 and 61.1; age 20 to 39 at 65.9 and 74.9; age 40 to 69 at 65.0 and 73.8; AGE 70 AND OVER AT 51.3 AND 60.8. EQUAL-OPPORTUNITY RATIO where 100 per cent is parity, baseline: gender 97.3 and 97.6

[FULL TEXT READ. Author order verified on the arXiv record and in the PDF: TIAN XU FIRST, University of Cambridge; White, Cambridge; Kalkan, Middle East Technical University; Gunes last, Cambridge] Xu, T., White, J., Kalkan, S., & Gunes, H. (2020). 'Investigating Bias and Fairness in Facial Expression Recognition.' arXiv:2007.10075v3. THE BEST-POWERED STUDY OF ACTUAL EXPRESSION RECOGNITION BY DEMOGRAPHIC GROUP, AND IT DOES NOT SAY WHAT PEOPLE EXPECT. Seven-way expression recognition on RAF-DB plus binary smiling on CelebA, with ResNet-18 baseline, attribute-aware and disentangled variants, each with and without augmentation. RAF-DB TEST SET COMPOSITION, Table 1, 2,876 test images of 14,388 total: race Caucasian 77.4 per cent, Asian 15.5 per cent, African-American 7.1 per cent; gender female 56.3 per cent, male 43.7 per cent; age 0 to 3 at 5.5 per cent, 4 to 19 at 16.4 per cent, 20 to 39 at 57.5 per cent, 40 to 69 at 17.4 per cent, 70 and over at 3.2 per cent; expression happy 39.7 per cent, neutral 22.6, sad 13.4, surprise 10.3, anger 5.7, disgust 5.5, fear 2.7. THE WITHIN-LABEL SKEW IS SHARPER THAN THE OVERALL SKEW AND IT IS THE DETAIL THAT MATTERS: of 164 anger images only 10 are African-American and 10 Asian; of 79 fear images, 5 are African-American. CLASS-WISE ACCURACY BY EXPRESSION, baseline without augmentation, mean 65.3 per cent: happy 91.6, neutral 78.7, surprise 75.8, anger 66.5, sad 63.2, disgust 41.1, fear 40.5. MEAN CLASS-WISE ACCURACY BY DEMOGRAPHIC GROUP, baseline, without and with augmentation: male 65.3 and 72.3; female 63.5 and 74.1; CAUCASIAN 65.9 AND 74.7; AFRICAN-AMERICAN 68.1 AND 76.3; ASIAN 60.0 AND 67.8; age 0 to 3 at 63.6 and 80.2; age 4 to 19 at 59.5 and 61.1; age 20 to 39 at 65.9 and 74.9; age 40 to 69 at 65.0 and 73.8; AGE 70 AND OVER AT 51.3 AND 60.8. EQUAL-OPPORTUNITY RATIO where 100 per cent is parity, baseline: gender 97.3 and 97.6

Written from the fULL TEXT READ. Author order verified on the arXiv record and in the PDF: TIAN XU FIRST, University of Cambridge; White, Cambridge; Kalkan, Middle East Technical University; Gunes last, Cambridge.

What our sources record

Recorded when this source was checked, figures included, with page references and quotation marks removed.

Xu, T., White, J., Kalkan, S., & Gunes, H. (2020). Investigating Bias and Fairness in Facial Expression Recognition. arXiv:2007.10075v3.