The FairCeptron: A Framework for Measuring Human Perceptions of Algorithmic Fairness

ACM Conference on User Modeling, Adaptation and Personalization (UMAP), pp. 401–403

Abstract

Measures of algorithmic fairness often do not account for human perceptions of fairness that can substantially vary between different sociodemographics and stakeholders. The FairCeptron framework is an approach for studying perceptions of fairness in algorithmic decision making such as in ranking or classification. It supports (i) studying human perceptions of fairness and (ii) comparing these human perceptions with measures of algorithmic fairness. The framework includes fairness scenario generation, fairness perception elicitation and fairness perception analysis. We demonstrate the FairCeptron framework by applying it to a hypothetical university admission context where we collect human perceptions of fairness in the presence of minorities. An implementation of the FairCeptron framework is openly available, and it can easily be adapted to study perceptions of algorithmic fairness in other application contexts. We hope our work paves the way towards elevating the role of studies of human fairness perceptions in the process of designing algorithmic decision making systems.

Authors 5

1. Georg Ahnert Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Germany

2. Ivan Smirnov Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Germany

3. Florian Lemmerich Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Germany

4. Claudia Wagner

   GESIS - Leibniz Institute for the Social Sciences

   Affiliation as printed

   GESIS - Leibniz Institute for the Social Sciences, Germany

5. Markus Strohmaier Aachen GESIS - Leibniz Institute for the Social Sciences

   GESIS - Leibniz Institute for the Social Sciences · RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, GESIS - Leibniz Institute for the Social Sciences, Germany