Fair Performance Metric Elicitation

06/23/2020
by   Gaurush Hiranandani, et al.
0

What is a fair performance metric? We consider the choice of fairness metrics through the lens of metric elicitation – a principled framework for selecting performance metrics that best reflect implicit preferences. The use of metric elicitation enables a practitioner to tune the performance and fairness metrics to the task, context, and population at hand. Specifically, we propose a novel strategy to elicit fair performance metrics for multiclass classification problems with multiple sensitive groups that also includes selecting the trade-off between performance and fairness. The proposed elicitation strategy requires only relative preference feedback and is robust to both finite sample and feedback noise.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset