TransCeption: Enhancing Medical Image Segmentation with an Inception-Like Transformer Design for Efficient Feature Fusion

Computational Visual Media, vol. 11, pp. 1079–1095

Abstract

While CNN-based methods have been the cornerstone of medical image segmentation due to their promising performance and robustness, they suffer from limitations in capturing long-range dependencies. Transformer-based approaches are currently prevailing since they enlarge the receptive field to model global contextual correlations. To further extract rich representations, some extensions of U-Net employ multi-scale feature extraction and fusion modules to obtain improved performance. Inspired by this idea, we propose TransCeption for medical image segmentation, a pure transformer-based U-shaped network incorporating an inception-like module in the encoder and adopting a contextual bridge for better feature fusion. The design proposed in this work is based on three core principles. (i) The patch merging module in the encoder is redesigned to use ResInception Patch Merging (RIPM). The MultiBranch (MB) transformer has the same number of branches as the outputs of RIPM. Combining the two modules enables the model to capture a multi-scale representation within a single stage. (ii) We apply an Intra-stage Feature Fusion (IFF) module following the MB transformer to enhance the aggregation of feature maps from all branches and particularly focus on the interaction between the different channels at all scales. (iii) In contrast to a bridge that only contains token-wise self-attention, we propose a Dual Transformer Bridge that also includes channel-wise self-attention to exploit correlations between scales at different stages from a dual perspective. Extensive experiments on multi-organ and skin lesion segmentation tasks show the superiority of TransCeption to previous work. The code is publicly available on GitHub.

Authors 5

1. Reza Azad Aachen Faculty of Electrical Engineering and Information Technology

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Faculty of Electrical Engineering and Information Technology,Aachen,Germany,52074

   Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen 52074, Germany

2. Yiwei Jia Aachen Faculty of Electrical Engineering and Information Technology

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Faculty of Electrical Engineering and Information Technology,Aachen,Germany,52074

   Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Aachen 52074, Germany

3. Ehsan Khodapanah Aghdam

   Shahid Beheshti University

   Affiliation as printed

   Shahid Beheshti University,Department of Electrical Engineering,Tehran,Iran,1983969411

   Department of Electrical Engineering, Shahid Beheshti University, Tehran 1983969411, Iran

4. Julien Cohen‐Adad

   Mila - Quebec Artificial Intelligence Institute · Polytechnique Montréal · Université de Montréal

   Affiliation as printed

   Institute of Biomedical Engineering, Polytechnique Montreal,NeuroPoly Lab,Montreal,Canada,H3T 1J4

   NeuroPoly Lab, Institute of Biomedical Engineering, Polytechnique Montreal, Montreal H3T 1J4, Canada; Functional Neuroimaging Unit, CRIUGM, University of Montreal, Montreal H3T 1J4, Canada; MILA, Quebec AI Institute, Montreal H2S 3H1, Canada

5. Dorit Merhof

   Fraunhofer Institute for Digital Medicine · University of Regensburg

   Affiliation as printed

   University of Regensburg,Faculty of Informatics and Data Science,Regensburg,Germany,93053

   Faculty of Informatics and Data Science, University of Regensburg, Regensburg 93053, Germany; Fraunhofer Institute for Digital Medicine MEVIS, Bremen 28359, Germany