Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification
[FULL TEXT READ at proceedings.mlr.press. Peer-reviewed conference proceedings, editors Sorelle A. Friedler and Christo Wilson, conference 23 to 24 February 2018] Buolamwini, J., & Gebru, T. (2018). 'Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification.' Proceedings of Machine Learning Research, 81, 77-91. SAY THIS PLAINLY IN THE ARTICLE: GENDER SHADES IS NOT AN EMOTION STUDY. It measures binary gender classification, whether a system labels a face male or female, and the only emotion content is a passing reference in related work. The authors state the generalisation question is open: 'Further work is needed to see if the substantial error rate gaps on the basis of gender, skin type and intersectional subgroup revealed in this study of gender classification persist in other human-based computer vision tasks.' SYSTEMS, THREE: Microsoft Cognitive Services Face API, IBM Watson Visual Recognition API and Face++, evaluated April and May 2017; the paper notes Google did not then provide a publicly available gender classifier. BENCHMARK: the Pilot Parliaments Benchmark, 1,270 unique individuals, parliamentarians from Rwanda 75, Senegal 149, South Africa 437, Sweden 349, Finland 197 and Iceland 63, official profile photographs with constrained pose and constant illumination, skin type labelled on the six-point Fitzpatrick scale with three annotators plus a board-certified surgical dermatologist providing the definitive labels; composition 44.6 per cent female, 46.4 per cent darker, with darker female 21.3 per cent, darker male 25.0 per cent, lighter female 23.3 per cent and lighter male 30.3 per cent. ERROR RATES, Table 4, error rate being one minus the true positive rate: MICROSOFT all 6.3 per cent, female 10.7, male 2.6, darker 12.9, lighter 0.7, DARKER FEMALE 20.8, darker male 6.0, lighter female 1.7, LIGHTER MALE 0.0; FACE++ all 10.0
[FULL TEXT READ at proceedings.mlr.press. Peer-reviewed conference proceedings, editors Sorelle A. Friedler and Christo Wilson, conference 23 to 24 February 2018] Buolamwini, J., & Gebru, T. (2018). 'Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification.' Proceedings of Machine Learning Research, 81, 77-91. SAY THIS PLAINLY IN THE ARTICLE: GENDER SHADES IS NOT AN EMOTION STUDY. It measures binary gender classification, whether a system labels a face male or female, and the only emotion content is a passing reference in related work. The authors state the generalisation question is open: 'Further work is needed to see if the substantial error rate gaps on the basis of gender, skin type and intersectional subgroup revealed in this study of gender classification persist in other human-based computer vision tasks.' SYSTEMS, THREE: Microsoft Cognitive Services Face API, IBM Watson Visual Recognition API and Face++, evaluated April and May 2017; the paper notes Google did not then provide a publicly available gender classifier. BENCHMARK: the Pilot Parliaments Benchmark, 1,270 unique individuals, parliamentarians from Rwanda 75, Senegal 149, South Africa 437, Sweden 349, Finland 197 and Iceland 63, official profile photographs with constrained pose and constant illumination, skin type labelled on the six-point Fitzpatrick scale with three annotators plus a board-certified surgical dermatologist providing the definitive labels; composition 44.6 per cent female, 46.4 per cent darker, with darker female 21.3 per cent, darker male 25.0 per cent, lighter female 23.3 per cent and lighter male 30.3 per cent. ERROR RATES, Table 4, error rate being one minus the true positive rate: MICROSOFT all 6.3 per cent, female 10.7, male 2.6, darker 12.9, lighter 0.7, DARKER FEMALE 20.8, darker male 6.0, lighter female 1.7, LIGHTER MALE 0.0; FACE++ all 10.0
Written from the fULL TEXT READ at proceedings.mlr.press. Peer-reviewed conference proceedings, editors Sorelle A. Friedler and Christo Wilson, conference 23 to 24 February 2018.
What our sources record
Recorded when this source was checked, figures included, with page references and quotation marks removed.
Buolamwini, J., & Gebru, T. (2018). Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. Proceedings of Machine Learning Research, 81, 77-91.