Beyond Self-Attention: Deformable Large Kernel Attention for Medical Image Segmentation

IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 1276–1286

Abstract

Medical image segmentation has seen significant improvements with transformer models, which excel in grasping far-reaching contexts and global contextual information. However, the increasing computational demands of these models, proportional to the squared token count, limit their depth and resolution capabilities. Most current methods process D volumetric image data slice-by-slice (called pseudo 3D), missing crucial inter-slice information and thus reducing the model’s overall performance. To address these challenges, we introduce the concept of Deformable Large Kernel Attention (D-LKA Attention), a streamlined attention mechanism employing large convolution kernels to fully appreciate volumetric context. This mechanism operates within a receptive field akin to self-attention while sidestepping the computational overhead. Additionally, our proposed attention mechanism benefits from deformable convolutions to flexibly warp the sampling grid, enabling the model to adapt appropriately to diverse data patterns. We designed both 2D and 3D adaptations of the D-LKA Attention, with the latter excelling in cross-depth data understanding. Together, these components shape our novel hierarchical Vision Transformer architecture, the D-LKA Net. Evaluations of our model against leading methods on popular medical segmentation datasets (Synapse, NIH Pancreas, and Skin lesion) demonstrate its superior performance. Our code is publicly available at GitHub.

Authors 8

1. Reza Azad Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University

2. Leon Niggemeier Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University

3. Michael Hüttemann Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University

4. Amirhossein Kazerouni

   Iran University of Science and Technology

   Affiliation as printed

   Iran University of Science and Technology

5. Ehsan Khodapanah Aghdam

   Shahid Beheshti University

   Affiliation as printed

   Shahid Beheshti University

6. Yury Velichko

   Northwestern University

   Affiliation as printed

   Northwestern University

7. Ulaş Bağcı

   Northwestern University

   Affiliation as printed

   Northwestern University

8. Dorit Merhof

   University of Regensburg

   Affiliation as printed

   University of Regensburg