Evaluating the human–artificial intelligence interaction in optical diagnosis of early colorectal carcinoma through endoscopist responses to simulated artificial intelligence: a multinational online study
Abstract
BACKGROUND: Suboptimal human-artificial intelligence (AI) interaction is a potential roadblock for implementation of AI in clinical practice. We aimed to evaluate the interaction between AI and endoscopists in the optical diagnosis of colorectal carcinoma (CRC). METHODS: International endoscopists were invited to diagnose colorectal lesions online. After a pretest of 15 videos, the diagnosis could be adjusted in the subsequent 35 videos after receiving a simulated AI diagnosis. Histology was the gold standard. Endoscopists' diagnoses before and after AI were compared using the signal detection theory. Inappropriate response to AI was defined as decreased diagnostic performance by the endoscopist after AI. RESULTS: < 0.001). Before AI, endoscopists missed only a few true CRCs (mean hit rate 0.86), at the cost of a mean false alarm rate of 0.35. Inappropriate response to AI was observed in 28.2 % of endoscopists. After AI, most endoscopists (41.0 %) shifted toward a more conservative attitude, tending to diagnose a lesion as non-CRC. CONCLUSIONS: Inappropriate response to AI is an issue in human-AI interaction for the optical diagnosis of CRC. Before using AI, endoscopists should become familiar with their own optical diagnostic performance, and that of the AI system, to possibly improve interaction.
Authors 27
-
Ayla Thijssen corresponding
Maastricht University Medical Centre · Maastricht University
Affiliation as printed
Department of Gastroenterology and Hepatology, Maastricht University Medical Center+, Maastricht, Netherlands
GROW Research Institute for Oncology and Reproduction, Maastricht University, Maastricht, Netherlands
-
Catharina Ziekenhuis · Maastricht University
Affiliation as printed
Department of Gastroenterology and Hepatology, Catharina Hospital, Eindhoven, Netherlands
GROW Research Institute for Oncology and Reproduction, Maastricht University, Maastricht, Netherlands
-
Eindhoven University of Technology
Affiliation as printed
Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
-
Amsterdam University Medical Centers
Affiliation as printed
Department of Gastroenterology and Hepatology, Amsterdam University Medical Centres, Amsterdam, Netherlands
-
Leiden University Medical Center
Affiliation as printed
Department of Gastroenterology and Hepatology, Leiden University Medical Center, Leiden, Netherlands
-
Affiliation as printed
Department of Gastroenterology and Hepatology, Bernhoven Hospital, Uden, Netherlands
-
Affiliation as printed
Department of Gastroenterology and Hepatology, Groene Hart Hospital, Gouda, Netherlands
-
Affiliation as printed
Department of Gastroenterology and Hepatology, Jeroen Bosch Hospital, Den Bosch, Netherlands
-
Affiliation as printed
Department of Gastroenterology and Hepatology, Gelderse Vallei Hospital, Ede, Netherlands
-
Affiliation as printed
Department of Gastroenterology and Hepatology, Isala Hospital, Zwolle, Netherlands
-
Affiliation as printed
Department of Gastroenterology and Hepatology, Maasstad Hospital, Rotterdam, Netherlands
-
Affiliation as printed
Department of Gastroenterology and Hepatology, Maxima Medical Center, Veldhoven, Netherlands
-
Radboud University Medical Center
Affiliation as printed
Department of Gastroenterology and Hepatology, Radboud University Medical Center, Nijmegen, Netherlands
-
Amsterdam University Medical Centers
Affiliation as printed
Department of Gastroenterology and Hepatology, Amsterdam University Medical Centres, Amsterdam, Netherlands
-
Maastricht University Medical Centre · Maastricht University
Affiliation as printed
Department of Gastroenterology and Hepatology, Maastricht University Medical Center+, Maastricht, Netherlands
-
Medisch Centrum Haaglanden · Stichting Haaglanden Medisch Centrum
Affiliation as printed
Department of Gastroenterology and Hepatology, Haaglanden Medical Center, Den Haag, Netherlands
-
Affiliation as printed
Department of Gastroenterology and Hepatology, St Jansdal Hospital, Harderwijk, Netherlands
-
Affiliation as printed
Department of Gastroenterology and Hepatology, Deventer Hospital, Deventer, Netherlands
-
The Netherlands Cancer Institute
Affiliation as printed
Department of Gastroenterology and Hepatology, Netherlands Cancer Institute, Amsterdam, Netherlands
-
Affiliation as printed
Department of Gastroenterology and Hepatology, Zuyderland Medical Center, Sittard-Geleen, Netherlands
-
Eindhoven University of Technology
Affiliation as printed
Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
-
Affiliation as printed
Department of Cognitive Neuroscience, Maastricht University, Maastricht, Netherlands
-
Maastricht University · Care and Public Health Research Institute
Affiliation as printed
Care and Public Health Research Institute (CAPHRI), Maastricht University, Maastricht, Netherlands
Department of Methodology and Statistics, Maastricht University, Maastricht, Netherlands
-
University Medical Center Utrecht
Affiliation as printed
Department of Gastroenterology and Hepatology, University Medical Center Utrecht, Utrecht, Netherlands
-
Eindhoven University of Technology
Affiliation as printed
Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, Netherlands
-
Catharina Ziekenhuis · Maastricht University
Affiliation as printed
Department of Gastroenterology and Hepatology, Catharina Hospital, Eindhoven, Netherlands
GROW Research Institute for Oncology and Reproduction, Maastricht University, Maastricht, Netherlands
Cited by 0 stored of 0
No patents citing this paper on Lens.org (checked 2026-10-11).