Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer
IEEE Access, vol. 9, pp. 95551–95568
Abstract
Medical image registration and segmentation are two of the most frequent tasks in medical image analysis. As these tasks are complementary and correlated, it would be beneficial to apply them simultaneously in a joint manner. In this paper, we formulate registration and segmentation as a joint problem via a Multi-Task Learning (MTL) setting, allowing these tasks to leverage their strengths and mitigate their weaknesses through the sharing of beneficial information. We propose to merge these tasks not only on the loss level, but on the architectural level as well. We studied this approach in the context of adaptive image-guided radiotherapy for prostate cancer, where planning and follow-up CT images as well as their corresponding contours are available for training. At testing time the contours of the follow-up scans are not available, which is a common scenario in adaptive radiotherapy. The study involves two datasets from different manufacturers and institutes. The first dataset was divided into training (12 patients) and validation (6 patients), and was used to optimize and validate the methodology, while the second dataset (14 patients) was used as an independent test set. We carried out an extensive quantitative comparison between the quality of the automatically generated contours from different network architectures as well as loss weighting methods. Moreover, we evaluated the quality of the generated deformation vector field (DVF). We show that MTL algorithms outperform their Single-Task Learning (STL) counterparts and achieve better generalization on the independent test set. The best algorithm achieved a mean surface distance of 1.06 ± 0.3 mm, 1.27 ± 0.4 mm, 0.91 ± 0.4 mm, and 1.76 ± 0.8 mm on the validation set for the prostate, seminal vesicles, bladder, and rectum, respectively. The high accuracy of the proposed method combined with the fast inference speed, makes it a promising method for automatic re-contouring of follow-up scans for adaptive radiotherapy, potentially reducing treatment related complications and therefore improving patients quality-of-life after treatment. The source code is available at https://github.com/moelmahdy/JRS-MTL.
Authors 7
-
Mohamed S. Elmahdy Aachen Department of Radiation Oncology Department of Radiology, Leiden University Medical Center
Leiden University Medical Center
Affiliation as printed
Department of Radiation Oncology, Leiden University Medical Center, Leiden, The Netherlands
Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
LEIDEN UNIVERSITY (MEDICAL CENTER)
-
Laurens Beljaards Aachen Department of Radiology, Leiden University Medical Center Leiden Institute of Advanced Computer Science
Leiden University · Leiden University Medical Center
Affiliation as printed
Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
Leiden Institute of Advanced Computer Science, Leiden, The Netherlands
-
Leiden University Medical Center
Affiliation as printed
Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
-
Leiden University Medical Center
Affiliation as printed
Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
-
Affiliation as printed
Leiden Institute of Advanced Computer Science, Leiden, The Netherlands
-
Leiden University Medical Center
Affiliation as printed
Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
-
Marius Staring Aachen Department of Radiation Oncology Department of Radiology, Leiden University Medical Center
Leiden University Medical Center · Delft University of Technology
Affiliation as printed
Department of Pattern Recognition and Bioinformatics, Delft University of Technology, Delft, The Netherlands
Department of Radiation Oncology, Leiden University Medical Center, Leiden, The Netherlands
Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
Cited by 35 stored of 35
Cited by patents worldwide 2 (Lens.org)
-
Jointly trained machine learning models for automatic contouring in radiotherapy applicationsUS12728284B2 2026-09-08 Active
-
JOINTLY TRAINED MACHINE LEARNING MODELS FOR AUTOMATIC CONTOURING IN RADIOTHERAPY APPLICATIONSUS20230377721A1 2023-11-23 Active