Towards fully automated inner ear analysis with deep-learning-based joint segmentation and landmark detection framework

Scientific Reports, vol. 13, pp. 19057

Abstract

Automated analysis of the inner ear anatomy in radiological data instead of time-consuming manual assessment is a worthwhile goal that could facilitate preoperative planning and clinical research. We propose a framework encompassing joint semantic segmentation of the inner ear and anatomical landmark detection of helicotrema, oval and round window. A fully automated pipeline with a single, dual-headed volumetric 3D U-Net was implemented, trained and evaluated using manually labeled in-house datasets from cadaveric specimen ([Formula: see text]) and clinical practice ([Formula: see text]). The model robustness was further evaluated on three independent open-source datasets ([Formula: see text] scans) consisting of cadaveric specimen scans. For the in-house datasets, Dice scores of [Formula: see text], intersection-over-union scores of [Formula: see text] and average Hausdorff distances of [Formula: see text] and [Formula: see text] voxel units were achieved. The landmark localization task was performed automatically with an average localization error of [Formula: see text] voxel units. A robust, albeit reduced performance could be attained for the catalogue of three open-source datasets. Results of the ablation studies with 43 mono-parametric variations of the basal architecture and training protocol provided task-optimal parameters for both categories. Ablation studies against single-task variants of the basal architecture showed a clear performance benefit of coupling landmark localization with segmentation and a dataset-dependent performance impact on segmentation ability.

Authors 6

1. Jannik Stebani corresponding

   University of Würzburg · Fraunhofer Institute for Integrated Circuits · Universitätsklinikum Würzburg

   Affiliation as printed

   Department of Oto-Rhino-Laryngology, Plastic, Aesthetic and Reconstructive Head and Neck Surgery and the Comprehensive Hearing Center, Universitätsklinikum Würzburg, 97080, Würzburg, Germany. jannik.stebani@iis.fraunhofer.de

   Magnetic Resonance and X-Ray Imaging Department, Fraunhofer Institute for Integrated Circuits IIS, 97074, Würzburg, Germany. jannik.stebani@iis.fraunhofer.de

   Universität Würzburg, Experimentelle Physik V, 97074, Würzburg, Germany. jannik.stebani@iis.fraunhofer.de

   Department of Oto-Rhino-Laryngology, Plastic, Aesthetic and Reconstructive Head and Neck Surgery and the Comprehensive Hearing Center, Universitätsklinikum Würzburg, Würzburg, Germany

   Magnetic Resonance and X-Ray Imaging Department, Fraunhofer Institute for Integrated Circuits IIS, Würzburg, Germany

   Universität Würzburg, Experimentelle Physik V, Würzburg, Germany

2. Martin Blaimer

   Fraunhofer Institute for Integrated Circuits

   Affiliation as printed

   Magnetic Resonance and X-Ray Imaging Department, Fraunhofer Institute for Integrated Circuits IIS, 97074, Würzburg, Germany

   Magnetic Resonance and X-Ray Imaging Department, Fraunhofer Institute for Integrated Circuits IIS, Würzburg, Germany

3. Simon Zabler

   Fraunhofer Institute for Integrated Circuits · Deggendorf Institute of Technology

   Affiliation as printed

   Faculty of Computer Science, Deggendorf Institute of Technology, Deggendorf, Germany

   Magnetic Resonance and X-Ray Imaging Department, Fraunhofer Institute for Integrated Circuits IIS, 97074, Würzburg, Germany

   Magnetic Resonance and X-Ray Imaging Department, Fraunhofer Institute for Integrated Circuits IIS, Würzburg, Germany

4. Tilmann Neun

   University of Würzburg · Universitätsklinikum Würzburg

   Affiliation as printed

   Institute for Diagnostic and Interventional Neuroradiology, Universitätsklinikum Würzburg, 97080, Würzburg, Germany

   Institute for Diagnostic and Interventional Neuroradiology, Universitätsklinikum Würzburg, Würzburg, Germany

5. Daniël M. Pelt Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden Institute of Advanced Computer Science (LIACS), Universiteit Leiden, Leiden, CA, 2333, The Netherlands

   Leiden Institute of Advanced Computer Science (LIACS), Universiteit Leiden, Leiden, The Netherlands

6. Kristen Rak

   University of Würzburg · Universitätsklinikum Würzburg

   Affiliation as printed

   Department of Oto-Rhino-Laryngology, Plastic, Aesthetic and Reconstructive Head and Neck Surgery and the Comprehensive Hearing Center, Universitätsklinikum Würzburg, 97080, Würzburg, Germany

   Department of Oto-Rhino-Laryngology, Plastic, Aesthetic and Reconstructive Head and Neck Surgery and the Comprehensive Hearing Center, Universitätsklinikum Würzburg, Würzburg, Germany