Computation-Efficient Era: A Comprehensive Survey of State Space Models in Medical Image Analysis

arXiv (Cornell University)

Abstract

Sequence modeling plays a vital role across various domains, with recurrent neural networks being historically the predominant method of performing these tasks. However, the emergence of transformers has altered this paradigm due to their superior performance. Built upon these advances, transformers have conjoined CNNs as two leading foundational models for learning visual representations. However, transformers are hindered by the $\mathcal{O}(N^2)$ complexity of their attention mechanisms, while CNNs lack global receptive fields and dynamic weight allocation. State Space Models (SSMs), specifically the \textit{\textbf{Mamba}} model with selection mechanisms and hardware-aware architecture, have garnered immense interest lately in sequential modeling and visual representation learning, challenging the dominance of transformers by providing infinite context lengths and offering substantial efficiency maintaining linear complexity in the input sequence. Capitalizing on the advances in computer vision, medical imaging has heralded a new epoch with Mamba models. Intending to help researchers navigate the surge, this survey seeks to offer an encyclopedic review of Mamba models in medical imaging. Specifically, we start with a comprehensive theoretical review forming the basis of SSMs, including Mamba architecture and its alternatives for sequence modeling paradigms in this context. Next, we offer a structured classification of Mamba models in the medical field and introduce a diverse categorization scheme based on their application, imaging modalities, and targeted organs. Finally, we summarize key challenges, discuss different future research directions of the SSMs in the medical domain, and propose several directions to fulfill the demands of this field. In addition, we have compiled the studies discussed in this paper along with their open-source implementations on our GitHub repository.

Authors 11

1. Moein Heidari

   University of British Columbia

   Affiliation as printed

   School of Biomedical Engineering University of British Columbia British Columbia , Canada

2. Sina Ghorbani Kolahi

   Tarbiat Modares University

   Affiliation as printed

   Department of Industrial and Systems Engineering Tarbiat Modares University Tehran , Iran

3. Sanaz Karimijafarbigloo

   OTH Regensburg · University of Regensburg

   Affiliation as printed

   Faculty of Informatics and Data Science University of Regensburg Regensburg , Germany

4. Bobby Azad

   South Dakota State University

   Affiliation as printed

   Electrical Engineering and Computer Science Department South Dakota State University Brookings , USA

5. Afshin Bozorgpour

   OTH Regensburg · University of Regensburg

   Affiliation as printed

   Faculty of Informatics and Data Science University of Regensburg Regensburg , Germany

6. Soheila Hatami

   Tarbiat Modares University

   Affiliation as printed

   Faculty of Mechanical Engineering Tarbiat Modares University Tehran , Iran

7. Reza Azad Aachen Faculty of Electrical Engineering and Information Technology

   RWTH Aachen University

   Affiliation as printed

   Faculty of Electrical Engineering and Information Technology RWTH Aachen University Aachen , Germany

8. Ali Diba
   Affiliation as printed

   Lunit Inc. Amsterdam , The Netherlands

9. Ulaş Bağcı

   Northwestern University · Northwestern University · Northwestern Medicine

   Affiliation as printed

   Department of Radiology Northwestern University Chicago , USA

10. Dorit Merhof

    OTH Regensburg · Fraunhofer Institute for Digital Medicine · University of Regensburg

    Affiliation as printed

    Faculty of Informatics and Data Science University of Regensburg Regensburg , Germany , and

    Fraunhofer Institute for Digital Medicine MEVIS , Bremen , Germany

11. Ilker Hacihaliloglu

    University of British Columbia

    Affiliation as printed

    Department of Radiology , Department of Medicine University of British Columbia British Columbia , Canada