Self-supervised Semantic Segmentation: Consistency over Transformation

IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), vol. 2023, pp. 2646–2655

Abstract

Accurate medical image segmentation is of utmost importance for enabling automated clinical decision procedures. However, prevailing supervised deep learning approaches for medical image segmentation encounter significant challenges due to their heavy dependence on extensive labeled training data. To tackle this issue, we propose a novel self-supervised algorithm, S3-Net, which integrates a robust framework based on the proposed Inception Large Kernel Attention (I-LKA) modules. This architectural enhancement makes it possible to comprehensively capture contextual information while preserving local intricacies, thereby enabling precise semantic segmentation. Furthermore, considering that lesions in medical images often exhibit deformations, we leverage deformable convolution as an integral component to effectively capture and delineate lesion deformations for superior object boundary definition. Additionally, our self-supervised strategy emphasizes the acquisition of invariance to affine transformations, which is commonly encountered in medical scenarios. This emphasis on robustness with respect to geometric distortions significantly enhances the model’s ability to accurately model and handle such distortions. To enforce spatial consistency and promote the grouping of spatially connected image pixels with similar feature representations, we introduce a spatial consistency loss term. This aids the network in effectively capturing the relationships among neighboring pixels and enhancing the overall segmentation quality. The S3-Net approach iteratively learns pixel-level feature representations for image content clustering in an end-to-end manner. Our experimental results on skin lesion and lung organ segmentation tasks show the superior performance of our method compared to the SOTA approaches. Github.

Authors 6

1. Sanaz Karimijafarbigloo

   University of Regensburg

   Affiliation as printed

   University of Regensburg,Faculty of Informatics and Data Science,Germany

   Faculty of Informatics and Data Science, University of Regensburg, Germany

2. Reza Azad Aachen Faculty of Electrical Engineering and Information Technology

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Faculty of Electrical Engineering and Information Technology,Germany

   Faculty of Electrical Engineering and Information Technology, RWTH Aachen University, Germany

3. Amirhossein Kazerouni

   Iran University of Science and Technology

   Affiliation as printed

   Iran University of Science and Technology,School of Electrical Engineering,Iran

   School of Electrical Engineering, Iran University of Science and Technology, Iran

4. Yury Velichko

   Northwestern University

   Affiliation as printed

   Northwestern University,Department of Radiology,Chicago,USA

   Department of Radiology, Northwestern University, Chicago, USA

5. Ulaş Bağcı

   Northwestern University

   Affiliation as printed

   Northwestern University,Department of Radiology,Chicago,USA

   Department of Radiology, Northwestern University, Chicago, USA

6. Dorit Merhof

   Fraunhofer Institute for Digital Medicine · University of Regensburg

   Affiliation as printed

   University of Regensburg,Faculty of Informatics and Data Science,Germany

   Faculty of Informatics and Data Science, University of Regensburg, Germany

   Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany