A Deep Learning-based in silico Framework for Optimization on Retinal Prosthetic Stimulation

Abstract

We propose a neural network-based framework to optimize the perceptions simulated by the in silico retinal implant model pulse2percept. The overall pipeline consists of a trainable encoder, a pre-trained retinal implant model and a pre-trained evaluator. The encoder is a U-Net, which takes the original image and outputs the stimulus. The pre-trained retinal implant model is also a U-Net, which is trained to mimic the biomimetic perceptual model implemented in pulse2percept. The evaluator is a shallow VGG classifier, which is trained with original images. Based on 10,000 test images from the MNIST dataset, we show that the convolutional neural network-based encoder performs significantly better than the trivial downsampling approach, yielding a boost in the weighted F1-Score by 36.17% in the pre-trained classifier with 6×10 electrodes. With this fully neural network-based encoder, the quality of the downstream perceptions can be fine-tuned using gradient descent in an end-to-end fashion.

Authors 5

1. Yuli Wu Aachen Institute of Imaging & Computer Vision

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Institute of Imaging and Computer Vision,Germany

   Institute of Imaging and Computer Vision, RWTH Aachen University, Germany

2. Ivan Karetić Aachen Institute of Imaging & Computer Vision

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Institute of Imaging and Computer Vision,Germany

   Institute of Imaging and Computer Vision, RWTH Aachen University, Germany

   NeuroTX Aachen e.V, Germany

3. Johannes Stegmaier Aachen Institute of Imaging & Computer Vision

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Institute of Imaging and Computer Vision,Germany

   Institute of Imaging and Computer Vision, RWTH Aachen University, Germany

4. Peter Walter Aachen Department of Ophthalmology

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Department of Ophthalmology,Germany

   Department of Ophthalmology, RWTH Aachen University, Germany

5. Dorit Merhof

   University of Regensburg

   Affiliation as printed

   University of Regensburg,Institute of Image Analysis and Computer Vision,Germany

   Institute of Image Analysis and Computer Vision, University of Regensburg, Germany