Local Implicit Neural Representations for Multi-Sequence MRI Translation

IEEE International Symposium on Biomedical Imaging (ISBI), pp. 1–5

Abstract

In radiological practice, multi-sequence MRI is routinely acquired to characterize anatomy and tissue. However, due to the heterogeneity of imaging protocols and contraindications to contrast agents, some MRI sequences, e.g. contrast-enhanced T1-weighted image (T1ce), may not be acquired. This creates difficulties for large-scale clinical studies for which heterogeneous datasets are aggregated. Modern deep learning techniques have demonstrated the capability of synthesizing missing sequences from existing sequences, through learning from an extensive multi-sequence MRI dataset. In this paper, we propose a novel MR image translation solution based on local implicit neural representations. We split the available MRI sequences into local patches and assign to each patch a local multi-layer perceptron (MLP) that represents a patch in the T1ce. The parameters of these local MLPs are generated by a hypernetwork based on image features. Experimental results and ablation studies on the BraTS challenge dataset showed that the local MLPs are critical for recovering fine image and tumor details, as they allow for local specialization that is highly important for accurate image translation. Compared to a classical pix2pix model, the proposed method demonstrated visual improvement and significantly improved quantitative scores (MSE 0.86 × 10−3vs. 1.02 × 10−3and SSIM 94.9 vs 94.3).

Authors 4

1. Yunjie Chen Aachen Department of Radiology, The Netherlands Department of Radiology, Leiden University Medical Center

   Leiden University Medical Center

   Affiliation as printed

   Leiden University Medical Center,Department of Radiology,The Netherlands

   Department of Radiology, Leiden University Medical Center, The Netherlands

2. Marius Staring Aachen Department of Radiology, The Netherlands Department of Radiology, Leiden University Medical Center

   Leiden University Medical Center

   Affiliation as printed

   Leiden University Medical Center,Department of Radiology,The Netherlands

   Department of Radiology, Leiden University Medical Center, The Netherlands

3. Jelmer M. Wolterink

   University of Twente

   Affiliation as printed

   University of Twente,Technical Medical Center,Department of Applied Mathematics,The Netherlands

   Department of Applied Mathematics, Technical Medical Center, University of Twente, The Netherlands

4. Qian Tao Aachen Department of Radiology, The Netherlands Department of Radiology, Leiden University Medical Center

   Leiden University Medical Center · Delft University of Technology

   Affiliation as printed

   Leiden University Medical Center,Department of Radiology,The Netherlands

   Department of Imaging Physics, Delft University of Technology, The Netherlands

   Department of Radiology, Leiden University Medical Center, The Netherlands