Algorithm appreciation or aversion: the effects of accuracy disclosure on users' reliance on algorithmic suggestions

Behaviour and Information Technology, pp. 1–20

Abstract

Prior work highlights people's tendency to initially overtrust algorithmic decisions, but also disproportionality lower their trust following an algorithm's mistakes. In contrast, people maintain higher trust when they see a human make similar mistakes. We evaluated how communicating accuracy information about an algorithm affected trust behaviour. Participants in Study I (N = 284) and Study II (N = 272) were given estimation tasks, and were provided with an algorithm's estimation as a suggestion. Half the participants received a description of the algorithm along with its average accuracy. During the task, algorithmic errors were introduced systematically, and participants noticed that the algorithm made substantial errors. Our results show that an algorithm's presentation significantly affects participants' reliance. Participants who were given information about the algorithm's accuracy conformed significantly more to the algorithm. Additionally, participants' reactions align with the previously documented algorithm aversion effect. Participants deviated more from the algorithm's estimations after observing the difference in performance between themselves and the algorithm. This resulted in reduced participant performance, as the algorithms were, on average, quite accurate. Furthermore, participants were able to distinguish between correct and incorrect suggestions, which is promising for successful trust calibration.

Authors 4

1. Sander de Jong corresponding

   Aalborg University

   Affiliation as printed

   Aalborg University

2. Maarten W. Bos

   Carnegie Mellon University

   Affiliation as printed

   Carnegie Mellon University

3. Niels van Berkel

   Aalborg University

   Affiliation as printed

   Aalborg University

4. Maarten H. Lamers Aachen

   Leiden University

   Affiliation as printed

   Leiden University