Overtrusting robots: Setting a research agenda to mitigate overtrust in automation

Paladyn Journal of Behavioral Robotics, vol. 12, pp. 423–436

Abstract

Abstract There is increasing attention given to the concept of trustworthiness for artificial intelligence and robotics. However, trust is highly context-dependent, varies among cultures, and requires reflection on others’ trustworthiness, appraising whether there is enough evidence to conclude that these agents deserve to be trusted. Moreover, little research exists on what happens when too much trust is placed in robots and autonomous systems. Conceptual clarity and a shared framework for approaching overtrust are missing. In this contribution, we offer an overview of pressing topics in the context of overtrust and robots and autonomous systems. Our review mobilizes insights solicited from in-depth conversations from a multidisciplinary workshop on the subject of trust in human–robot interaction (HRI), held at a leading robotics conference in 2020. A broad range of participants brought in their expertise, allowing the formulation of a forward-looking research agenda on overtrust and automation biases in robotics and autonomous systems. Key points include the need for multidisciplinary understandings that are situated in an eco-system perspective, the consideration of adjacent concepts such as deception and anthropomorphization, a connection to ongoing legal discussions through the topic of liability, and a socially embedded understanding of overtrust in education and literacy matters. The article integrates diverse literature and provides a ground for common understanding for overtrust in the context of HRI.

Authors 10

1. Alexander Mois Aroyo

   University of Waterloo

   Affiliation as printed

   SIRRL, University of Waterloo , Waterloo , Canada

2. Orian Dheu

   KU Leuven

   Affiliation as printed

   KU Leuven Centre for IT & IP Law (CiTiP) , KU Leuven, Leuven , Belgium

3. Eduard Fosch‐Villaronga Aachen eLaw Center for Law and Digital Technologies

   Leiden University

   Affiliation as printed

   eLaw Center for Law and Digital Technologies, Leiden University , Leiden , The Netherlands

4. Aleksei Gudkov

   National Research University Higher School of Economics

   Affiliation as printed

   Faculty of Law: School for Theory of Law and Cross-sectoral Legal Disciplines, National Research University Higher School of Economics , Moscow , Russia

5. Holly Hoch

   University of St.Gallen

   Affiliation as printed

   FAA-HSG, University of St. Gallen , St. Gallen , Switzerland

6. Steve Jones

   University of Illinois Chicago

   Affiliation as printed

   Department of Communication, University of Illinois Chicago , Chicago , United States

7. Christoph Lutz corresponding

   BI Norwegian Business School

   Affiliation as printed

   Nordic Centre for Internet and Society, BI Norwegian Business School , Nydalsveien 37 , 0484 Oslo , Norway

8. Henrik Skaug Sætra

   Østfold University of Applied Sciences

   Affiliation as printed

   Department of Computer Science and Communication, Østfold University College , Østfold , Norway

9. Mads Solberg

   Norwegian University of Science and Technology

   Affiliation as printed

   Department of Health Sciences in Ålesund, Norwegian University of Science and Technology , Trondheim , Norway

10. Aurelia Tamò‐Larrieux

    University of St.Gallen

    Affiliation as printed

    FAA-HSG, University of St. Gallen , St. Gallen , Switzerland