MRSegmentator: Multi-Modality Segmentation of 40 Classes in MRI and CT

arXiv (Cornell University)

Abstract

Purpose: To develop and evaluate a deep learning model for multi-organ segmentation of MRI scans. Materials and Methods: The model was trained on 1,200 manually annotated 3D axial MRI scans from the UK Biobank, 221 in-house MRI scans, and 1228 CT scans from the TotalSegmentator dataset. A human-in-the-loop annotation workflow was employed, leveraging cross-modality transfer learning from an existing CT segmentation model to segment 40 anatomical structures. The annotation process began with a model based on transfer learning between CT and MR, which was iteratively refined based on manual corrections to predicted segmentations. The model's performance was evaluated on MRI examinations obtained from the German National Cohort (NAKO) study (n=900) from the AMOS22 dataset (n=60) and from the TotalSegmentator-MRI test data (n=29). The Dice Similarity Coefficient (DSC) and Hausdorff Distance (HD) were used to assess segmentation quality, stratified by organ and scan type. The model and its weights will be open-sourced. Results: MRSegmentator demonstrated high accuracy for well-defined organs (lungs: DSC 0.96, heart: DSC 0.94) and organs with anatomic variability (liver: DSC 0.96, kidneys: DSC 0.95). Smaller structures showed lower accuracy (portal/splenic veins: DSC 0.64, adrenal glands: DSC 0.69). On external validation using NAKO data, mean DSC ranged from 0.85 $\pm$ 0.08 for T2-HASTE to 0.91 $\pm$ 0.05 for in-phase sequences. The model generalized well to CT, achieving mean DSC of 0.84 $\pm$ 0.11 on AMOS CT data. Conclusion: MRSegmentator accurately segments 40 anatomical structures in MRI across diverse datasets and imaging protocols, with additional generalizability to CT images. This open-source model will provide a valuable tool for automated multi-organ segmentation in medical imaging research. It can be downloaded from https://github.com/hhaentze/MRSegmentator.

Authors 15

1. Hartmut Häntze

   Humboldt-Universität zu Berlin · Charité - Universitätsmedizin Berlin · Radboud University Medical Center · Freie Universität Berlin

   Affiliation as printed

   Department of Radiology , Charité -Universitätsmedizin Berlin corporate member of Freie Universität Berlin and

   Diagnostic Image Analysis Group , Radboud University Medical Center , The Netherlands

2. Lina Xu

   Humboldt-Universität zu Berlin · Charité - Universitätsmedizin Berlin · Freie Universität Berlin

   Affiliation as printed

   Department of Radiology , Charité -Universitätsmedizin Berlin corporate member of Freie Universität Berlin and

3. Felix J. Dorfner

   Humboldt-Universität zu Berlin · Charité - Universitätsmedizin Berlin · Freie Universität Berlin · Athinoula A. Martinos Center for Biomedical Imaging

   Affiliation as printed

   Athinoula A. Martinos Center for Biomedical Imaging , Massachusetts General Hospital and Harvard Medical School , USA

   Department of Radiology , Charité -Universitätsmedizin Berlin corporate member of Freie Universität Berlin and

4. Leonhard Donle

   Humboldt-Universität zu Berlin · Charité - Universitätsmedizin Berlin · Freie Universität Berlin

   Affiliation as printed

   Department of Radiology , Charité -Universitätsmedizin Berlin corporate member of Freie Universität Berlin and

5. Felix Busch

   TUM Klinikum

   Affiliation as printed

   Department of Diagnostic and Interventional Radiology , School of Medicine and Health , Klinikum rechts der Isar , TUM University Hospital , Germany

6. Sebastian Ziegelmayer

   TUM Klinikum

   Affiliation as printed

   Department of Diagnostic and Interventional Radiology , School of Medicine and Health , Klinikum rechts der Isar , TUM University Hospital , Germany

7. Nadine Bayerl

   Friedrich-Alexander-Universität Erlangen-Nürnberg

   Affiliation as printed

   Institute of Radiology , University Hospital Erlangen , Friedrich-Alexander-Universität (FAU) Erlangen-Nürnberg , Germany

8. Hugo J.W.L. Aerts
9. Daniel Truhn Aachen Department of Diagnostic and Interventional Radiology

   Universitätsklinikum Aachen

   Affiliation as printed

   Department of Diagnostic and Interventional Radiology , University Hospital Aachen , Germany

10. Klaus Hermann Maier-Hein

    DKFZ-ZMBH Alliance

    Affiliation as printed

    Division of Medical Image Computing , Deutsches Krebsforschungszentrum Heidelberg , Germany

11. Mathias Prokop

    Humboldt-Universität zu Berlin · Radboud University Medical Center

    Affiliation as printed

    Diagnostic Image Analysis Group , Radboud University Medical Center , The Netherlands

    Humboldt Universität zu Berlin , Germany

12. Bram van Ginneken

    Radboud University Medical Center

    Affiliation as printed

    Diagnostic Image Analysis Group , Radboud University Medical Center , The Netherlands

13. Alessa Hering

    Radboud University Medical Center

    Affiliation as printed

    Diagnostic Image Analysis Group , Radboud University Medical Center , The Netherlands

14. Lisa C. Adams

    TUM Klinikum

    Affiliation as printed

    Department of Diagnostic and Interventional Radiology , School of Medicine and Health , Klinikum rechts der Isar , TUM University Hospital , Germany

15. Keno K. Bressem

    TUM Klinikum · Deutsches Herzzentrum München

    Affiliation as printed

    Department of Cardiovascular Radiology and Nuclear Medicine , School of Medicine and Health , German Heart Center , TUM University Hospital , Technical University of Munich , Germany

    Department of Diagnostic and Interventional Radiology , School of Medicine and Health , Klinikum rechts der Isar , TUM University Hospital , Germany