AIROGS: Artificial Intelligence for Robust Glaucoma Screening Challenge
IEEE Transactions on Medical Imaging, vol. 43, pp. 542–557
Abstract
The early detection of glaucoma is essential in preventing visual impairment. Artificial intelligence (AI) can be used to analyze color fundus photographs (CFPs) in a cost-effective manner, making glaucoma screening more accessible. While AI models for glaucoma screening from CFPs have shown promising results in laboratory settings, their performance decreases significantly in real-world scenarios due to the presence of out-of-distribution and low-quality images. To address this issue, we propose the Artificial Intelligence for Robust Glaucoma Screening (AIROGS) challenge. This challenge includes a large dataset of around 113,000 images from about 60,000 patients and 500 different screening centers, and encourages the development of algorithms that are robust to ungradable and unexpected input data. We evaluated solutions from 14 teams in this paper and found that the best teams performed similarly to a set of 20 expert ophthalmologists and optometrists. The highest-scoring team achieved an area under the receiver operating characteristic curve of 0.99 (95% CI: 0.98-0.99) for detecting ungradable images on-the-fly. Additionally, many of the algorithms showed robust performance when tested on three other publicly available datasets. These results demonstrate the feasibility of robust AI-enabled glaucoma screening.
Authors 32
-
Radboud University Nijmegen · Amsterdam UMC Location University of Amsterdam · Radboud University Medical Center · Amsterdam University Medical Centers · University of Amsterdam
Affiliation as printed
Quantitative Healthcare Analysis (QurAI) Group, Informatics Institute, Universiteit van Amsterdam, Amsterdam, The Netherlands
Department of Biomedical Engineering and Physics, Amsterdam UMC Locatie AMC, Amsterdam, Noord-Holland, Netherlands
Department of Radiology and Nuclear Medicine, Diagnostic Image Analysis Group (DIAG), Radboudumc, Nijmegen, Gelderland, Netherlands
Informatics Institute, Quantitative Healthcare Analysis (QurAI) Group, Universiteit van Amsterdam, Amsterdam, Noord-Holland, Netherlands
-
Affiliation as printed
Rotterdam Ophthalmic Institute, The Rotterdam Eye Hospital, Rotterdam, The Netherlands
-
Affiliation as printed
Project Orbis International Inc., New York, NY, USA
Project Orbis International Inc, New York, United States
-
Capital Medical University · Peking Union Medical College Hospital · Xuan Wu Hospital of the Capital Medical University
Affiliation as printed
Peking Union Medical College Hospital, Beijing, China
Xuanwu Hospital Capital Medical University, Beijing, China
-
Affiliation as printed
Department of Automation, Tsinghua University, Beijing, China
-
Affiliation as printed
Department of Diagnostic and Interventional Radiology, University Hospital Aachen, Aachen, Germany
-
Affiliation as printed
Department of Diagnostic and Interventional Radiology, University Hospital Aachen, Aachen, Germany
-
Affiliation as printed
CMC Technologies LLP, Nur-Sultan, Kazakhstan
-
Affiliation as printed
CMC Technologies LLP, Nur-Sultan, Kazakhstan
-
Universitat Pompeu Fabra · The University of Adelaide
Affiliation as printed
Departament de Tecnologies de la Informació i les Comunicacions (DTIC), Universitat Pompeu Fabra, Barcelona, Spain
Australian Institute for Machine Learning AIML, University of Adelaide, Australia
-
University of Surrey · The University of Adelaide
Affiliation as printed
Australian Institute for Machine Learning AIML, University of Adelaide, Adelaide, SA, Australia
Centre for Vision, Speech and Signal Processing, University of Surrey, United Kingdom
-
College of Engineering Trivandrum
Affiliation as printed
College of Engineering Trivandrum, Thiruvananthapuram, India
-
Affiliation as printed
Founding Minds Software, Thiruvananthapuram, India
-
Affiliation as printed
Founding Minds Software, Thiruvananthapuram, India
-
Chinese Academy of Sciences · Institute of Computing Technology
Affiliation as printed
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
-
Chinese Academy of Sciences · Institute of Computing Technology
Affiliation as printed
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
-
Muroran Institute of Technology
Affiliation as printed
Muroran Institute of Technology, Muroran, Japan
-
Niigata University of Health and Welfare
Affiliation as printed
Niigata University of Health and Welfare, Niigata, Japan
-
Affiliation as printed
Schulich School of Medicine and Dentistry, University of Western Ontario, London, Canada
-
Affiliation as printed
Department of Mathematics and Computer Science, University of La Rioja, Logroño, Spain
-
Vall d'Hebron Hospital Universitari
Affiliation as printed
Hospital Vall Hebron, Sant Cugat del Vallés, Barcelona, Spain
-
Affiliation as printed
Department of Ophthalmology and Optometry, Christian Doppler Laboratory for Artificial Intelligence in Retina, Medical University of Vienna, Vienna, Austria
-
Affiliation as printed
Department of Ophthalmology and Optometry, Christian Doppler Laboratory for Artificial Intelligence in Retina, Medical University of Vienna, Vienna, Austria
-
Affiliation as printed
Department of Ophthalmology and Optometry, Christian Doppler Laboratory for Artificial Intelligence in Retina, Medical University of Vienna, Vienna, Austria
-
Samsung Medical Center · Research Institute for Future Medicine
Affiliation as printed
Research Institute for Future Medicine, Samsung Medical Center, Seoul, Republic of Korea
-
Samsung Medical Center · Sungkyunkwan University
Affiliation as printed
Department of Digital Health, Samsung Advanced Institute for Health Sciences and Technology (SAIHST), Sungkyunkwan University, Samsung Medical Center, Seoul, Republic of Korea
-
Samsung Medical Center · Mayo Clinic in Florida · Sungkyunkwan University
Affiliation as printed
Department of Digital Health, Samsung Advanced Institute for Health Sciences and Technology (SAIHST), Sungkyunkwan University, Samsung Medical Center, Seoul, Republic of Korea
Department of Artificial Intelligence and Informatics, Mayo Clinic, Jacksonville, Florida, United States of America
-
Affiliation as printed
Department of Biomedical Engineering, King’s College London, London, U.K
-
Affiliation as printed
School of Computer Science and Engineering, University of New South Wales, Sydney, NSW, Australia
-
Radboud University Nijmegen · Radboud University Medical Center
Affiliation as printed
Department of Radiology and Nuclear Medicine, Diagnostic Image Analysis Group (DIAG), Radboudumc, Nijmegen, The Netherlands
Department of Radiology and Nuclear Medicine, Diagnostic Image Analysis Group (DIAG), Radboudumc, Nijmegen, Gelderland, Netherlands
-
Affiliation as printed
Rotterdam Ophthalmic Institute, The Rotterdam Eye Hospital, Rotterdam, The Netherlands
-
Amsterdam UMC Location University of Amsterdam · Amsterdam University Medical Centers · University of Amsterdam
Affiliation as printed
Quantitative Healthcare Analysis (QurAI) Group, Informatics Institute, Universiteit van Amsterdam, Amsterdam, The Netherlands
Department of Biomedical Engineering and Physics, Amsterdam UMC Locatie AMC, Amsterdam, Noord-Holland, Netherlands
Informatics Institute, Quantitative Healthcare Analysis (QurAI) Group, Universiteit van Amsterdam, Amsterdam, Noord-Holland, Netherlands
Cited by 87 stored of 89
Cited by patents worldwide 1 (Lens.org)
-
PROTOTYPE-BASED TASK INDEPENDENT INTERPRETABLE MODELEP4651094A1 2025-11-19 Pending