FairFace Challenge at ECCV 2020: Analyzing Bias in Face Recognition

by   Tomáš Sixta, et al.

This work summarizes the 2020 ChaLearn Looking at People Fair Face Recognition and Analysis Challenge and provides a description of the top-winning solutions and analysis of the results. The aim of the challenge was to evaluate accuracy and bias in gender and skin colour of submitted algorithms on the task of 1:1 face verification in the presence of other confounding attributes. Participants were evaluated using an in-the-wild dataset based on reannotated IJB-C, further enriched by 12.5K new images and additional labels. The dataset is not balanced, which simulates a real world scenario where AI-based models supposed to present fair outcomes are trained and evaluated on imbalanced data. The challenge attracted 151 participants, who made more than 1.8K submissions in total. The final phase of the challenge attracted 36 active teams out of which 10 exceeded 0.999 AUC-ROC while achieving very low scores in the proposed bias metrics. Common strategies by the participants were face pre-processing, homogenization of data distributions, the use of bias aware loss functions and ensemble models. The analysis of top-10 teams shows higher false positive rates (and lower false negative rates) for females with dark skin tone as well as the potential of eyeglasses and young age to increase the false positive rates too.


page 14

page 26


Bias Mitigation of Face Recognition Models Through Calibration

Face recognition models suffer from bias: for example, the probability o...

Meta Balanced Network for Fair Face Recognition

Although deep face recognition has achieved impressive progress in recen...

Comparing Human and Machine Bias in Face Recognition

Much recent research has uncovered and discussed serious concerns of bia...

OCFR 2022: Competition on Occluded Face Recognition From Synthetically Generated Structure-Aware Occlusions

This work summarizes the IJCB Occluded Face Recognition Competition 2022...

Consistent Instance False Positive Improves Fairness in Face Recognition

Demographic bias is a significant challenge in practical face recognitio...

Reducing Geographic Performance Differential for Face Recognition

As face recognition algorithms become more accurate and get deployed mor...

Characterizing the Variability in Face Recognition Accuracy Relative to Race

Many recent news headlines have labeled face recognition technology as b...

Please sign up or login with your details

Forgot password? Click here to reset