Leveraging Vision Transformers with Hyperparameter Optimization for the Classification of Acute Respiratory Distress Syndrome

MIPRO ICT and Electronics Convention (MIPRO), pp. 1293–1298

Abstract

Acute respiratory distress syndrome (ARDS) is a serious lung condition associated with a high mortality rate. The classification of this condition poses a challenge in intensive care medicine and diagnostic imaging. Artificial intelligence methods, particularly deep learning, can assist the diagnostic process. Recent advances have demonstrated the potential of vision transformers to improve image analysis through their ability to extract relevant features and complex patterns in images. In this study, a vision transformer is implemented for classifying ARDS in chest X-Rays using a two-step transfer learning approach. For this purpose, publicly available databases of X-rays are used, some of which have been annotated by a radiologist for the use case ARDS. Furthermore, in order to uncover the optimal combination of model parameters to streamline the training process, we implement a two-tier hyperparameter optimization using the Ray Tune framework on high-performance computing infrastructure. The retrained vision transformer was able to classify ARDS data with$\mathbf{9 5 \%}$accuracy, outperforming previous approaches employing residual networks. Our results highlight the improvement that can be achieved through a two-step transfer learning approach leveraging vision transformers and taking advantage of powerful supercomputing architecture. Ultimately, our work facilitates timely and accurate classification of ARDS thereby enabling improved outcomes for patients receiving critical care.

Authors 7

1. Josefine Busch

   Forschungszentrum Jülich · Jülich Supercomputing Centre

   Affiliation as printed

   Jülich Supercomputing Centre,Forschungszentrum Jülich,Germany

2. Chadi Barakat

   Forschungszentrum Jülich · Jülich Supercomputing Centre

   Affiliation as printed

   Jülich Supercomputing Centre,Forschungszentrum Jülich,Germany

3. T. Pauli Aachen Informatik 11 - Chair of Embedded Software

   RWTH Aachen University

   Affiliation as printed

   Informatik 11 - Chair of Embedded Software, RWTH Aachen University,Germany

4. Simon Fonck Aachen Informatik 11 - Chair of Embedded Software

   RWTH Aachen University

   Affiliation as printed

   Informatik 11 - Chair of Embedded Software, RWTH Aachen University,Germany

5. André Stollenwerk Aachen Informatik 11 - Chair of Embedded Software

   RWTH Aachen University

   Affiliation as printed

   Informatik 11 - Chair of Embedded Software, RWTH Aachen University,Germany

6. Sebastian Johannes Fritsch

   Forschungszentrum Jülich · Jülich Supercomputing Centre

   Affiliation as printed

   Jülich Supercomputing Centre,Forschungszentrum Jülich,Germany

7. Morris Riedel

   Forschungszentrum Jülich · Jülich Supercomputing Centre

   Affiliation as printed

   Jülich Supercomputing Centre,Forschungszentrum Jülich,Germany