Overcoming Data Scarcity in Biomedical Imaging with a Foundational Multi-Task Model

arXiv (Cornell University)

Abstract

Foundational models, pretrained on a large scale, have demonstrated substantial success across non-medical domains. However, training these models typically requires large, comprehensive datasets, which contrasts with the smaller and more heterogeneous datasets common in biomedical imaging. Here, we propose a multi-task learning strategy that decouples the number of training tasks from memory requirements. We trained a Universal bioMedical PreTrained model (UMedPT) on a multi-task database including tomographic, microscopic, and X-ray images, with various labelling strategies such as classification, segmentation, and object detection. The UMedPT foundational model outperformed ImageNet pretraining and the previous state-of-the-art models. For tasks related to the pretraining database, it maintained its performance with only 1% of the original training data and without fine-tuning. For out-of-domain tasks it required not more than 50% of the original training data. In an external independent validation imaging features extracted using UMedPT proved to be a new standard for cross-center transferability.

Authors 9

1. Raphael Schäfer
2. Till Nicke

   Fraunhofer Institute for Digital Medicine

   Affiliation as printed

   Fraunhofer Institute for Digital Medicine MEVIS

3. Henning Höfener
4. Annkristin Lange

   Fraunhofer Institute for Digital Medicine

   Affiliation as printed

   Fraunhofer Institute for Digital Medicine MEVIS

5. Dorit Merhof

   Fraunhofer Institute for Digital Medicine · University of Regensburg

   Affiliation as printed

   Fraunhofer Institute for Digital Medicine MEVIS

   Institute of Image Analysis and Computer Vision , Faculty of Informatics and Data Science , University of Regensburg

6. Friedrich Feuerhake

   University Medical Center Freiburg · Medizinische Hochschule Hannover

   Affiliation as printed

   Institute for Neuropathology , University Clinic Freiburg

   Institute for Pathology , Hannover Medical School

7. Volkmar Schulz Aachen Institute for Experimental Molecular Imaging

   Fraunhofer Institute for Digital Medicine · RWTH Aachen University

   Affiliation as printed

   Fraunhofer Institute for Digital Medicine MEVIS

   Institute for Experimental Molecular Imaging , RWTH Aachen University

8. Johannes Lotz

   Fraunhofer Institute for Digital Medicine

   Affiliation as printed

   Fraunhofer Institute for Digital Medicine MEVIS

9. Fabian M. Kiessling Aachen Institute for Experimental Molecular Imaging

   Fraunhofer Institute for Digital Medicine · RWTH Aachen University

   Affiliation as printed

   Fraunhofer Institute for Digital Medicine MEVIS

   Institute for Experimental Molecular Imaging , RWTH Aachen University