Conditional Generative Adversarial Networks for Data Augmentation of a Neonatal Image Dataset

Sensors, vol. 23, pp. 999

Abstract

In today's neonatal intensive care units, monitoring vital signs such as heart rate and respiration is fundamental for neonatal care. However, the attached sensors and electrodes restrict movement and can cause medical-adhesive-related skin injuries due to the immature skin of preterm infants, which may lead to serious complications. Thus, unobtrusive camera-based monitoring techniques in combination with image processing algorithms based on deep learning have the potential to allow cable-free vital signs measurements. Since the accuracy of deep-learning-based methods depends on the amount of training data, proper validation of the algorithms is difficult due to the limited image data of neonates. In order to enlarge such datasets, this study investigates the application of a conditional generative adversarial network for data augmentation by using edge detection frames from neonates to create RGB images. Different edge detection algorithms were used to validate the input images' effect on the adversarial network's generator. The state-of-the-art network architecture Pix2PixHD was adapted, and several hyperparameters were optimized. The quality of the generated RGB images was evaluated using a Mechanical Turk-like multistage survey conducted by 30 volunteers and the FID score. In a fake-only stage, 23% of the images were categorized as real. A direct comparison of generated and real (manually augmented) images revealed that 28% of the fake data were evaluated as more realistic. An FID score of 103.82 was achieved. Therefore, the conducted study shows promising results for the training and application of conditional generative adversarial networks to augment highly limited neonatal image datasets.

Authors 6

1. Simon Lyra corresponding Aachen Medical Information Technology (MedIT)

   RWTH Aachen University

   Affiliation as printed

   Medical Information Technology, Helmholtz Institute for Biomedical Engineering, RWTH Aachen University, 52074 Aachen, Germany

2. Arian Mustafa Aachen Medical Information Technology (MedIT)

   RWTH Aachen University

   Affiliation as printed

   Medical Information Technology, Helmholtz Institute for Biomedical Engineering, RWTH Aachen University, 52074 Aachen, Germany

3. Jöran Rixen Aachen Medical Information Technology (MedIT)

   RWTH Aachen University

   Affiliation as printed

   Medical Information Technology, Helmholtz Institute for Biomedical Engineering, RWTH Aachen University, 52074 Aachen, Germany

4. Štefan Borik

   University of Žilina

   Affiliation as printed

   Department of Electromagnetic and Biomedical Engineering, Faculty of Electrical Engineering and Information Technology, University of Zilina, 010 26 Zilina, Slovakia

5. Markus Lueken Aachen Medical Information Technology (MedIT)

   RWTH Aachen University

   Affiliation as printed

   Medical Information Technology, Helmholtz Institute for Biomedical Engineering, RWTH Aachen University, 52074 Aachen, Germany

6. Steffen K. Leonhardt Aachen Medical Information Technology (MedIT)

   RWTH Aachen University

   Affiliation as printed

   Medical Information Technology, Helmholtz Institute for Biomedical Engineering, RWTH Aachen University, 52074 Aachen, Germany