Understanding metric-related pitfalls in image analysis validation
Nature Methods, vol. 21, pp. 182–194
Authors 69
-
Annika Reinke corresponding
German Cancer Research Center · Heidelberg University
Affiliation as printed
Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany. a.reinke@dkfz-heidelberg.de
German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany. a.reinke@dkfz-heidelberg.de
German Cancer Research Center (DKFZ) Heidelberg, HI Helmholtz Imaging, Heidelberg, Germany. a.reinke@dkfz-heidelberg.de
German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
-
Minu D. Tizabi corresponding
German Cancer Research Center · National Center for Tumor Diseases
Affiliation as printed
German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany. m.tizabi@dkfz-heidelberg.de
National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany. m.tizabi@dkfz-heidelberg.de
German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
-
Affiliation as printed
German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Heidelberg, Germany
-
Affiliation as printed
German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
-
German Cancer Research Center · National Center for Tumor Diseases
Affiliation as printed
German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany
-
Affiliation as printed
German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Heidelberg, Germany
German Cancer Research Center (DKFZ) Heidelberg, HI Applied Computer Vision Lab, Heidelberg, Germany
-
Affiliation as printed
German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
German Cancer Research Center (DKFZ) Heidelberg, HI Helmholtz Imaging, Heidelberg, Germany
-
King's College London · MRC Unit for Lifelong Health and Ageing · University College London
Affiliation as printed
MRC Unit for Lifelong Health and Ageing at UCL and Centre for Medical Image Computing, Department of Computer Science, University College London, London, UK
School of Biomedical Engineering and Imaging Science, King's College London, London, UK
-
Consejo Nacional de Investigaciones Científicas y Técnicas · Universidad de Buenos Aires
Affiliation as printed
Instituto de Cálculo, CONICET - Universidad de Buenos Aires, Buenos Aires, Argentina
Instituto de Cálculo, CONICET – Universidad de Buenos Aires, Buenos Aires, Argentina
-
King's College London · University College London
Affiliation as printed
Centre for Medical Image Computing, University College London, London, UK
School of Biomedical Engineering and Imaging Science, King's College London, London, UK
-
Mila - Quebec Artificial Intelligence Institute · McGill University
Affiliation as printed
Centre for Intelligent Machines and MILA (Quebec Artificial Intelligence Institute), McGill University, Montréal, Quebec, Canada
-
Indiana University Indianapolis · Indiana University School of Medicine · University of Pennsylvania
Affiliation as printed
Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
Division of Computational Pathology, Dept of Pathology & Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, USA
-
Affiliation as printed
Department of Digital Medical Technologies, Holon Institute of Technology, Holon, Israel
European Federation for Medical Informatics, Le Mont-sur-Lausanne, Switzerland
-
Goethe University Frankfurt · German Cancer Research Center · Frankfurt Cancer Institute
Affiliation as printed
Frankfurt Cancer Insititute, Frankfurt am Main, Germany
German Cancer Consortium (DKTK), partner site Frankfurt/Mainz, a partnership between DKFZ and UCT Frankfurt-Marburg, Frankfurt am Main, Germany
German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
Goethe University Frankfurt, Department of Informatics, Frankfurt am Main, Germany
Goethe University Frankfurt, Department of Medicine, Frankfurt am Main, Germany
-
Affiliation as printed
School of Biomedical Engineering and Imaging Science, King's College London, London, UK
-
Affiliation as printed
Department of Computer Science, IT University of Copenhagen, Copenhagen, Denmark
-
Leibniz Institute for Analytical Sciences - ISAS
Affiliation as printed
Leibniz-Institut für Analytische Wissenschaften - ISAS - e.V., Dortmund, Germany
Leibniz-Institut für Analytische Wissenschaften – ISAS – e.V., Dortmund, Germany
-
Affiliation as printed
German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
-
Affiliation as printed
Imaging Platform, Broad Institute of MIT and Harvard, Cambridge, MA, USA
-
Affiliation as printed
Center for Biomedical Informatics and Information Technology, National Cancer Institute, Bethesda, MD, USA
-
Consejo Nacional de Investigaciones Científicas y Técnicas · Universidad de Buenos Aires
Affiliation as printed
Instituto de Investigación en Ciencias de la Computación (ICC), CONICET-UBA, Ciudad Autónoma de Buenos Aires, Buenos Aires, Argentina
-
Universitat Pompeu Fabra · The University of Adelaide
Affiliation as printed
Universitat Pompeu Fabra, Barcelona, Spain
University of Adelaide, Adelaide, South Australia, Australia
-
Radboud University Nijmegen · Radboud University Medical Center · Fraunhofer Institute for Digital Medicine · Radboud Institute for Health Sciences
Affiliation as printed
Fraunhofer MEVIS, Bremen, Germany
Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, the Netherlands
-
Affiliation as printed
Department of Computing, Imperial College London, South Kensington Campus, London, UK
-
German Cancer Research Center · Heidelberg University · National Center for Tumor Diseases
Affiliation as printed
Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany
-
Affiliation as printed
Department of Surgery, Perelman School of Medicine, Philadelphia, PA, USA
General Robotics Automation Sensing and Perception Laboratory, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA, USA
-
University Health Network · University of Toronto · Princess Margaret Cancer Centre · Vector Institute
Affiliation as printed
Department of Computer Science, University of Toronto, Toronto, Ontario, Canada
Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
Princess Margaret Cancer Centre, University Health Network, Toronto, Ontario, Canada
Vector Institute for Artificial Intelligence, Toronto, Ontario, Canada
-
Radboud University Medical Center
Affiliation as printed
Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
-
Affiliation as printed
German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Heidelberg, Germany
German Cancer Research Center (DKFZ) Heidelberg, HI Applied Computer Vision Lab, Heidelberg, Germany
-
Inserm · Laboratoire Traitement du Signal et de l'Image · Université Rennes 1 · Université de Rennes
Affiliation as printed
INSERM, Paris, France
Laboratoire Traitement du Signal et de l'Image - UMR_S 1099, Université de Rennes 1, Rennes, France
Laboratoire Traitement du Signal et de l'Image – UMR_S 1099, Université de Rennes 1, Rennes, France
-
Affiliation as printed
Department of Radiology and Institute for Biomedical Informatics, University of Pennsylvania, Philadelphia, PA, USA
-
University of Potsdam · Max Delbrück Center
Affiliation as printed
Max-Delbrück Center for Molecular Medicine in the Helmholtz Association (MDC), Biomedical Image Analysis and HI Helmholtz Imaging, Berlin, Germany
University of Potsdam, Digital Engineering Faculty, Potsdam, Germany
-
Friedrich-Alexander-Universität Erlangen-Nürnberg · Imperial College London
Affiliation as printed
Department AIBE, Friedrich-Alexander-Universität (FAU), Erlangen-Nürnberg, Germany
Department of Computing, Faculty of Engineering, Imperial College London, London, UK
-
Institut de Chirurgie Guidée par l'Image
Affiliation as printed
IHU Strasbourg, Strasbourg, France
-
Essen University Hospital · Stiftung Universitätsmedizin Essen
Affiliation as printed
Translational Image-guided Oncology (TIO), Institute for AI in Medicine (IKIM), University Medicine Essen, Essen, Germany
-
Affiliation as printed
Helmholtz AI, Oberschleißheim, Germany
-
Affiliation as printed
Lunit, Seoul, South Korea
-
Affiliation as printed
German Cancer Research Center (DKFZ) Heidelberg, Division of Biostatistics, Heidelberg, Germany
-
Affiliation as printed
Centre for Biomedical Image Analysis and Faculty of Informatics, Masaryk University, Brno, Czech Republic
-
European Molecular Biology Laboratory
Affiliation as printed
Cell Biology and Biophysics Unit, European Molecular Biology Laboratory (EMBL), Heidelberg, Germany
-
Affiliation as printed
Department of Biomedical Informatics, Stony Brook University, Health Science Center, Stony Brook, NY, USA
-
Affiliation as printed
Electrical Engineering, Vanderbilt University, Nashville, TN, USA
-
Radboud University Medical Center
Affiliation as printed
Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
-
Affiliation as printed
Department of Surgery, University Health Network, Philadelphia, PA, USA
-
Klaus Hermann Maier-Hein corresponding
German Cancer Research Center · Heidelberg University · University Hospital Heidelberg
Affiliation as printed
German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Heidelberg, Germany
Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany
-
University of Toronto · Sunnybrook Hospital · Sunnybrook Research Institute
Affiliation as printed
Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
Physical Sciences, Sunnybrook Research Institute, Toronto, Ontario, Canada
-
Affiliation as printed
School of Computer Science and Engineering, University of New South Wales, UNSW Sydney, Kensington, New South Wales, Australia
-
Affiliation as printed
Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland
-
Utrecht University · University Medical Center Utrecht
Affiliation as printed
Julius Center for Health Sciences and Primary Care, UMC Utrecht, Utrecht University, Utrecht, the Netherlands
-
University of Geneva · HES-SO University of Applied Sciences and Arts Western Switzerland
Affiliation as printed
Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO), Sierre, Switzerland
Medical Faculty, University of Geneva, Geneva, Switzerland
-
Mila - Quebec Artificial Intelligence Institute
Affiliation as printed
MILA (Quebec Artificial Intelligence Institute), Montréal, Quebec, Canada
-
University Medical Center Hamburg-Eppendorf
Affiliation as printed
Department of General, Visceral and Thoracic Surgery, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
-
Affiliation as printed
German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Heidelberg, Germany
-
Allen Institute for Cell Science
Affiliation as printed
Allen Institute for Cell Science, Seattle, WA, USA
-
University of Bern · University Hospital of Bern
Affiliation as printed
ARTORG Center for Biomedical Engineering Research, University of Bern, Bern, Switzerland
Department of Radiation Oncology, University Hospital Bern, University of Bern, Bern, Switzerland
-
Simula Metropolitan Center for Digital Engineering · UiT The Arctic University of Norway
Affiliation as printed
Simula Metropolitan Center for Digital Engineering, Oslo, Norway
UiT The Arctic University of Norway, Tromsø, Norway
-
Affiliation as printed
NVIDIA GmbH, München, Germany
-
Heidelberg University · University Hospital Heidelberg
Affiliation as printed
Faculty of Medicine, Heidelberg University Hospital, Heidelberg, Germany
Institute for Computational Biomedicine, Heidelberg University, Heidelberg, Germany
-
Affiliation as printed
Informatics Institute, Faculty of Science, University of Amsterdam, Amsterdam, the Netherlands
-
Affiliation as printed
Google Health, Google, Palo Alto, CA, USA
-
National Institutes of Health · National Institutes of Health Clinical Center
Affiliation as printed
National Institutes of Health Clinical Center, Bethesda, MD, USA
-
Affiliation as printed
Institute of Information Systems Engineering, TU Wien, Vienna, Austria
-
Oulu University Hospital · University of Oulu
Affiliation as printed
Neurocenter Oulu, Oulu University Hospital, Oulu, Finland
Research Unit of Health Sciences and Technology, Faculty of Medicine, University of Oulu, Oulu, Finland
-
Affiliation as printed
School of Engineering, The University of Edinburgh, Edinburgh, Scotland
-
Leiden University Medical Center · KU Leuven
Affiliation as printed
Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands
Department of Development and Regeneration and EPI-centre, KU Leuven, Leuven, Belgium
-
Institut national de recherche en sciences et technologies du numérique · Centre Inria de Saclay
Affiliation as printed
Parietal project team, INRIA Saclay-Île de France, Palaiseau, France
-
National Institute of Allergy and Infectious Diseases
Affiliation as printed
National Institute of Allergy and Infectious Diseases, Bethesda, MD, USA
-
Paul F. Jäger corresponding
Affiliation as printed
German Cancer Research Center (DKFZ) Heidelberg, HI Helmholtz Imaging, Heidelberg, Germany. p.jaeger@dkfz-heidelberg.de
German Cancer Research Center (DKFZ) Heidelberg, Interactive Machine Learning Group, Heidelberg, Germany. p.jaeger@dkfz-heidelberg.de
German Cancer Research Center (DKFZ) Heidelberg, Interactive Machine Learning Group, Heidelberg, Germany
-
Lena Maier‐Hein corresponding
German Cancer Research Center · Heidelberg University · University Hospital Heidelberg · National Center for Tumor Diseases
Affiliation as printed
Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany. l.maier-hein@dkfz-heidelberg.de
Faculty of Medicine, Heidelberg University Hospital, Heidelberg, Germany. l.maier-hein@dkfz-heidelberg.de
German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany. l.maier-hein@dkfz-heidelberg.de
German Cancer Research Center (DKFZ) Heidelberg, HI Helmholtz Imaging, Heidelberg, Germany. l.maier-hein@dkfz-heidelberg.de
National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany. l.maier-hein@dkfz-heidelberg.de
Faculty of Medicine, Heidelberg University Hospital, Heidelberg, Germany
Cited by 206 stored of 209
No patents citing this paper on Lens.org (checked 2026-10-11).