Evaluating the Explainable AI Method Grad-CAM for Breath Classification on Newborn Time Series Data

IFAC-PapersOnLine, vol. 58, pp. 123–128

Abstract

With the digitalization of health care systems, artificial intelligence becomes more present in medicine. Especially machine learning shows great potential for complex tasks such as time series classification, usually at the cost of transparency and comprehensibility. This leads to a lack of trust by humans and thus hinders its active usage. Explainable artificial intelligence tries to close this gap by providing insight into the decision-making process, the actual usefulness of its different methods is however unclear. This paper proposes a user study based evaluation of the explanation method Grad-CAM with application to a neural network for the classification of breaths in time series neonatal ventilation data. We present the perceived usefulness of the explainability method by different stakeholders, exposing the difficulty to achieve actual transparency and the wish for more in-depth explanations by many of the participants.

Authors 8

1. Camelia Oprea corresponding Aachen Chair for Embedded Software

   RWTH Aachen University

   Affiliation as printed

   Chair for Embedded Software, RWTH Aachen University, Aachen, Germany

   Co-First Authors

2. Mike Grüne Aachen Chair for Embedded Software

   RWTH Aachen University

   Affiliation as printed

   Chair for Embedded Software, RWTH Aachen University, Aachen, Germany

   Co-First Authors

3. Mateusz Buglowski Aachen Chair for Embedded Software

   RWTH Aachen University

   Affiliation as printed

   Chair for Embedded Software, RWTH Aachen University, Aachen, Germany

4. Lena Olivier Aachen Neonatology Section of the Department of Paediatric and Adolescent Medicine

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Neonatology Section of the Department of Paediatric and Adolescent Medicine, RWTH Aachen University Hospital, Aachen, Germany

5. Thorsten W Orlikowsky Aachen Neonatology Section of the Department of Paediatric and Adolescent Medicine

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Neonatology Section of the Department of Paediatric and Adolescent Medicine, RWTH Aachen University Hospital, Aachen, Germany

6. Stefan Kowalewski Aachen Chair for Embedded Software

   RWTH Aachen University

   Affiliation as printed

   Chair for Embedded Software, RWTH Aachen University, Aachen, Germany

7. Mark Schoberer Aachen Neonatology Section of the Department of Paediatric and Adolescent Medicine

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Equally contributing

   Neonatology Section of the Department of Paediatric and Adolescent Medicine, RWTH Aachen University Hospital, Aachen, Germany

8. André Stollenwerk Aachen Chair for Embedded Software

   RWTH Aachen University

   Affiliation as printed

   Chair for Embedded Software, RWTH Aachen University, Aachen, Germany

   Equally contributing