Fairness-Aware Ranking Recovery from Pairwise Comparisons

ACM Web Science Conference (WebSci), pp. 289–298

Abstract

Pairwise comparisons based on human judgments are widely used to derive rankings, yet the biases embedded in these comparisons can propagate into downstream, algorithmically generated rankings. Such bias can reduce the visibility of underprivileged groups, with potentially profound consequences. In this work we analyze two points in the ranking pipeline where bias can emerge: (i) the selection of candidates that receive comparisons, and (ii) the comparison and ranking mechanisms themselves. To assess fairness in this context, we introduce a group-conditioned accuracy metric that captures how ranking quality varies across demographic groups.

Authors 3

1. Georg Ahnert

   University of Mannheim

   Affiliation as printed

   University of Mannheim, Mannheim, Germany

2. Antonio Ferrara

   Graz University of Technology

   Affiliation as printed

   Intesa Sanpaolo AI Research, Turin, Italy and Graz University of Technology, Graz, Austria

3. Claudia Wagner Aachen GESIS - Leibniz Institute for the Social Sciences

   RWTH Aachen University · GESIS - Leibniz Institute for the Social Sciences

   Affiliation as printed

   GESIS–Leibniz Institute for the Social Sciences, Cologne, Germany and RWTH Aachen University, Aachen, Germany