Overcoming data scarcity in biomedical imaging with a foundational multi-task model

Nature Computational Science, vol. 4, pp. 495–509

Abstract

Foundational models, pretrained on a large scale, have demonstrated substantial success across non-medical domains. However, training these models typically requires large, comprehensive datasets, which contrasts with the smaller and more specialized datasets common in biomedical imaging. Here we propose a multi-task learning strategy that decouples the number of training tasks from memory requirements. We trained a universal biomedical pretrained model (UMedPT) on a multi-task database including tomographic, microscopic and X-ray images, with various labeling strategies such as classification, segmentation and object detection. The UMedPT foundational model outperformed ImageNet pretraining and previous state-of-the-art models. For classification tasks related to the pretraining database, it maintained its performance with only 1% of the original training data and without fine-tuning. For out-of-domain tasks it required only 50% of the original training data. In an external independent validation, imaging features extracted using UMedPT proved to set a new standard for cross-center transferability.

Authors 8

1. Till Nicke

   Fraunhofer Institute for Digital Medicine

   Affiliation as printed

   Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany

2. Henning Höfener

   Fraunhofer Institute for Digital Medicine

   Affiliation as printed

   Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany

3. Annkristin Lange

   Fraunhofer Institute for Digital Medicine

   Affiliation as printed

   Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany

4. Dorit Merhof

   Fraunhofer Institute for Digital Medicine · University of Regensburg

   Affiliation as printed

   Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany

   Institute of Image Analysis and Computer Vision, Faculty of Informatics and Data Science, University of Regensburg, Regensburg, Germany

5. Friedrich Feuerhake

   University of Freiburg · University Medical Center Freiburg · Medizinische Hochschule Hannover

   Affiliation as printed

   Institute for Neuropathology, Medical Center, University of Freiburg, Freiburg, Germany

   Institute for Pathology, Hannover Medical School, Hanover, Germany

6. Volkmar Schulz Aachen Institute for Experimental Molecular Imaging

   Fraunhofer Institute for Digital Medicine · RWTH Aachen University

   Affiliation as printed

   Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany

   Institute for Experimental Molecular Imaging, RWTH Aachen University, Aachen, Germany

7. Johannes Lotz corresponding

   Fraunhofer Institute for Digital Medicine

   Affiliation as printed

   Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany. johannes.lotz@mevis.fraunhofer.de

   Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany

8. Fabian M. Kiessling corresponding Aachen Institute for Experimental Molecular Imaging

   Fraunhofer Institute for Digital Medicine · RWTH Aachen University

   Affiliation as printed

   Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany. fkiessling@ukaachen.de

   Institute for Experimental Molecular Imaging, RWTH Aachen University, Aachen, Germany. fkiessling@ukaachen.de

   Institute for Experimental Molecular Imaging, RWTH Aachen University, Aachen, Germany