Smartphone video nystagmography using convolutional neural networks: ConVNG
Journal of Neurology, vol. 270, pp. 2518–2530
Abstract
BACKGROUND: Eye movement abnormalities are commonplace in neurological disorders. However, unaided eye movement assessments lack granularity. Although videooculography (VOG) improves diagnostic accuracy, resource intensiveness precludes its broad use. To bridge this care gap, we here validate a framework for smartphone video-based nystagmography capitalizing on recent computer vision advances. METHODS: A convolutional neural network was fine-tuned for pupil tracking using > 550 annotated frames: ConVNG. In a cross-sectional approach, slow-phase velocity of optokinetic nystagmus was calculated in 10 subjects using ConVNG and VOG. Equivalence of accuracy and precision was assessed using the "two one-sample t-test" (TOST) and Bayesian interval-null approaches. ConVNG was systematically compared to OpenFace and MediaPipe as computer vision (CV) benchmarks for gaze estimation. RESULTS: ConVNG tracking accuracy reached 9-15% of an average pupil diameter. In a fully independent clinical video dataset, ConVNG robustly detected pupil keypoints (median prediction confidence 0.85). SPV measurement accuracy was equivalent to VOG (TOST p < 0.017; Bayes factors (BF) > 24). ConVNG, but not MediaPipe, achieved equivalence to VOG in all SPV calculations. Median precision was 0.30°/s for ConVNG, 0.7°/s for MediaPipe and 0.12°/s for VOG. ConVNG precision was significantly higher than MediaPipe in vertical planes, but both algorithms' precision was inferior to VOG. CONCLUSIONS: ConVNG enables offline smartphone video nystagmography with an accuracy comparable to VOG and significantly higher precision than MediaPipe, a benchmark computer vision application for gaze estimation. This serves as a blueprint for highly accessible tools with potential to accelerate progress toward precise and personalized Medicine.
Authors 8
-
Maximilian Uwe Friedrich corresponding
Affiliation as printed
Department of Neurology, University Hospital Wuerzburg, Josef-Schneider Strasse 11, 97080, Würzburg, Germany. Friedrich_M6@ukw.de
Department of Neurology, University Hospital Wuerzburg, Josef-Schneider Strasse 11, 97080, Würzburg, Germany
-
Brandenburg University of Technology Cottbus-Senftenberg
Affiliation as printed
Brandenburg University of Technology Cottbus, Senftenberg, Germany
-
University of Würzburg · Universitätsklinikum Würzburg
Affiliation as printed
Department of Otorhinolaryngology, University Hospital Wuerzburg, Josef-Schneider Strasse 11, 97080, Würzburg, Germany
-
Affiliation as printed
Department of Otorhinolaryngology, University Hospital Aachen, Pauwelsstraße 30, 52074, Aachen, Germany
-
Affiliation as printed
Department of Neurology, University Hospital Wuerzburg, Josef-Schneider Strasse 11, 97080, Würzburg, Germany
-
Affiliation as printed
Department of Neurology, University Hospital Wuerzburg, Josef-Schneider Strasse 11, 97080, Würzburg, Germany
-
Robert Lucien Peach corresponding
Engineering and Physical Sciences Research Council · Universitätsklinikum Würzburg · Imperial College London
Affiliation as printed
Department of Brain Sciences and EPSRC Centre for Mathematics of Precision Healthcare, Imperial College London, London, SW7 2AZ, UK. Peach_R@ukw.de
Department of Neurology, University Hospital Wuerzburg, Josef-Schneider Strasse 11, 97080, Würzburg, Germany. Peach_R@ukw.de
Department of Brain Sciences and EPSRC Centre for Mathematics of Precision Healthcare, Imperial College London, London, SW7 2AZ, UK
-
Affiliation as printed
Department of Neurology, University Hospital Wuerzburg, Josef-Schneider Strasse 11, 97080, Würzburg, Germany
Cited by 44 stored of 44
Cited by patents worldwide 1 (Lens.org)
-
DETECTION AND EVALUATION OF PUPIL MOVEMENTSWO2024175410A1 2024-08-29 Pending