MSA$^2$Net: Multi-scale Adaptive Attention-guided Network for Medical Image Segmentation

arXiv (Cornell University)

Abstract

Medical image segmentation involves identifying and separating object instances in a medical image to delineate various tissues and structures, a task complicated by the significant variations in size, shape, and density of these features. Convolutional neural networks (CNNs) have traditionally been used for this task but have limitations in capturing long-range dependencies. Transformers, equipped with self-attention mechanisms, aim to address this problem. However, in medical image segmentation it is beneficial to merge both local and global features to effectively integrate feature maps across various scales, capturing both detailed features and broader semantic elements for dealing with variations in structures. In this paper, we introduce MSA$^2$Net, a new deep segmentation framework featuring an expedient design of skip-connections. These connections facilitate feature fusion by dynamically weighting and combining coarse-grained encoder features with fine-grained decoder feature maps. Specifically, we propose a Multi-Scale Adaptive Spatial Attention Gate (MASAG), which dynamically adjusts the receptive field (Local and Global contextual information) to ensure that spatially relevant features are selectively highlighted while minimizing background distractions. Extensive evaluations involving dermatology, and radiological datasets demonstrate that our MSA$^2$Net outperforms state-of-the-art (SOTA) works or matches their performance. The source code is publicly available at https://github.com/xmindflow/MSA-2Net.

Authors 8

1. Sina Ghorbani Kolahi

   Tarbiat Modares University

   Affiliation as printed

   Tarbiat Modares University Tehran , Iran

2. S. Kamal Chaharsooghi

   Tarbiat Modares University

   Affiliation as printed

   Tarbiat Modares University Tehran , Iran

3. Toktam Khatibi

   Tarbiat Modares University

   Affiliation as printed

   Tarbiat Modares University Tehran , Iran

4. Afshin Bozorgpour

   University of Regensburg

   Affiliation as printed

   University of Regensburg Regensburg , Germany

5. Reza Azad Aachen

   University of Regensburg · RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University Aachen , Germany

   University of Regensburg Regensburg , Germany

6. Moein Heidari

   University of British Columbia

   Affiliation as printed

   University of British Columbia British Columbia , Canada

7. Ilker Hacihaliloglu

   University of British Columbia

   Affiliation as printed

   University of British Columbia British Columbia , Canada

8. Dorit Merhof

   Fraunhofer Institute for Digital Medicine · University of Regensburg

   Affiliation as printed

   Fraunhofer Institute for Digital Medicine MEVIS Bremen , Germany

   University of Regensburg Regensburg , Germany