Beyond benchmarking: an expert-guided consensus approach to spatially aware clustering
Nature Methods, vol. 23, pp. 1814–1826
Abstract
Spatial omics technologies have revolutionized the study of tissue architecture and cellular heterogeneity by integrating molecular profiles with spatial localization. In spatially resolved transcriptomics, delineating higher-order anatomical structures is critical for understanding how cellular organization affects tissue and organ function. Since 2020, more than 50 spatially aware clustering (SAC) methods have been developed for this purpose. However, the reliability of current benchmarks is undermined by their narrow focus on Visium and brain tissue datasets, as well as incorrect interpretation of manual annotation as ground truth. Here, we present SACCELERATOR, a community-driven, extensible framework that standardizes data formatting, method integration, and metric evaluation, and is designed to rapidly incorporate new methods and datasets. SACCELERATOR currently includes 22 SAC methods applied to 15 datasets spanning 9 technologies and diverse tissue types. Our analysis revealed substantial limitations in the generalizability and reproducibility of SAC methods across tissues and platforms. We also demonstrate that anatomical labels commonly used as ground truths are often biased, potentially error-prone, and, in some cases, unsuitable for benchmarking efforts. Rather than scoring and comparing methods, we propose a consensus-guided workflow that aggregates clustering results to generate consensus representations. Descriptive spatial metrics highlight areas of high entropy where method disagreement is highest, enabling targeted feedback for tissue experts. Applied to brain and cancer datasets, this approach uncovered biologically meaningful patterns overlooked by individual methods and manual annotations. Our results underscore the need for iterative, expert-in-the-loop analysis and reveal that traditional evaluation metrics do not always capture the subjective qualities of results. By improving tissue annotation and addressing key benchmarking limitations, SACCELERATOR provides a robust foundation for advancing spatial omics research.
Authors 54
-
Centre Hospitalier Universitaire Vaudois
Affiliation as printed
Biomedical Data Science Center, Centre Hospitalier Universitaire Vaudois, Lausanne, Switzerland
Biomedical Data Science Center, Centre Hospitalier Universitaire Vaudois, Rue du Bugnon 2, 0 Lausanne, Switzerland
-
Leiden University Medical Center · Delft University of Technology
Affiliation as printed
Delft Bioinformatics Lab, Delft University of Technology, Delft, the Netherlands
Department of Human Genetics, Leiden University Medical Center, Leiden, the Netherlands
Delft Bioinformatics Lab, Delft University of Technology, Van Mourik Broekmanweg 6, 2628 XE Delft, Netherlands
Department of Human Genetics, Leiden University Medical Center, Einthovenweg 0, 333 ZC Leiden, Netherlands
-
SIB Swiss Institute of Bioinformatics · University of Zurich
Affiliation as printed
Department of Molecular Life Sciences and SIB Swiss Institute of Bioinformatics, University of Zurich, Zurich, Switzerland
Department of Molecular Life Sciences and SIB Swiss Institute of Bioinformatics, University of Zurich, Winterthurerstrasse 190, 8057 Zurich, Switzerland
-
Berlin Institute of Health at Charité - Universitätsmedizin Berlin
Affiliation as printed
Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Center of Digital, Berlin, Germany
Berlin Institute of Health at Charité – Universitätsmedizin Berlin, Center of Digital Health, Charitéplatz 1, 10117 Berlin, Germany
-
Universitätsklinikum Aachen · RWTH Aachen University
Affiliation as printed
Department of Nephrology, Rheumatology, and Clinical Immunology, University Hospital RWTH Aachen, Aachen, Germany
Department of Nephrology, Rheumatology, and Clinical Immunology, University Hospital RWTH Aachen, Pauwelsstraße 30, Aachen, Germany
-
Allen Institute for Brain Science
Affiliation as printed
Allen Institute for Brain Science, Seattle, WA, USA
Allen Institute for Brain Science, 615 Westlake Ave N, Seattle, WA, USA 98109
-
University of Copenhagen · LEO Foundation · Technical University of Denmark
Affiliation as printed
DTU Health Tech, Technical University of Denmark, Ørsteds Plads, Kongens Lyngby, Denmark
LEO Foundation Skin Immunology Research Center, Department of Immunology and Microbiology, University of Copenhagen, Copenhagen, Denmark
-
Affiliation as printed
German Cancer Research Center (DKFZ), Division of Computational Genomics and Systems Genetics, Heidelberg, Germany
The German Human Genome-Phenome Archive, Heidelberg, Germany
German Cancer Research Center (DKFZ), Division of Computational Genomics and Systems Genetics, Im Neuenheimer Feld 280, 69120 Heidelberg, Germany
-
SIB Swiss Institute of Bioinformatics · University of Zurich
Affiliation as printed
Department of Molecular Life Sciences and SIB Swiss Institute of Bioinformatics, University of Zurich, Zurich, Switzerland
Department of Molecular Life Sciences and SIB Swiss Institute of Bioinformatics, University of Zurich, Winterthurerstrasse 190, 8057 Zurich, Switzerland
-
SIB Swiss Institute of Bioinformatics · University of Zurich
Affiliation as printed
Department of Molecular Life Sciences and SIB Swiss Institute of Bioinformatics, University of Zurich, Zurich, Switzerland
Department of Molecular Life Sciences and SIB Swiss Institute of Bioinformatics, University of Zurich, Winterthurerstrasse 190, 8057 Zurich, Switzerland
-
SIB Swiss Institute of Bioinformatics · University of Zurich
Affiliation as printed
Department of Molecular Life Sciences and SIB Swiss Institute of Bioinformatics, University of Zurich, Zurich, Switzerland
Department of Molecular Life Sciences and SIB Swiss Institute of Bioinformatics, University of Zurich, Winterthurerstrasse 190, 8057 Zurich, Switzerland
-
Berlin Institute of Health at Charité - Universitätsmedizin Berlin
Affiliation as printed
Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Center of Digital, Berlin, Germany
Berlin Institute of Health at Charité – Universitätsmedizin Berlin, Center of Digital Health, Charitéplatz 1, 10117 Berlin, Germany
-
Affiliation as printed
Division of Cell Biology, Histology and Embryology, Gottfried Schatz Research Center, Medical University of Graz, Graz, Austria
-
University of Graz · Medical University of Graz
Affiliation as printed
Diagnostic and Research Institute of Pathology, Medical University of Graz, Graz, Austria
-
Delft University of Technology
Affiliation as printed
Delft Bioinformatics Lab, Delft University of Technology, Van Mourik Broekmanweg 6, 2628 XE Delft, Netherlands
-
University of Michigan · Centre Hospitalier Universitaire Vaudois · Michigan Medicine
Affiliation as printed
Department of Internal Medicine, Division of Nephrology, University of Michigan, Ann Arbor, MI, USA
Biomedical Data Science Center, Centre Hospitalier Universitaire Vaudois, Rue du Bugnon 2, 0 Lausanne, Switzerland
-
Universitätsklinikum Aachen · Carnegie Mellon University · RWTH Aachen University
Affiliation as printed
Computational Biology Department, Carnegie Mellon University, Pittsburgh, PA, USA
Department of Nephrology, Rheumatology, and Clinical Immunology, University Hospital RWTH Aachen, Pauwelsstraße 30, Aachen, Germany
-
Allen Institute for Brain Science · Centro de Investigación Médica Aplicada
Affiliation as printed
CIMA Universidad de Navarra, Pamplona, Spain
Allen Institute for Brain Science, 615 Westlake Ave N, Seattle, WA, USA 98109
-
Leiden University Medical Center · The Ohio State University · Delft University of Technology
Affiliation as printed
Department of Biomedical Informatics, College of Medicine, Ohio State University, Columbus, OH, USA
Delft Bioinformatics Lab, Delft University of Technology, Van Mourik Broekmanweg 6, 2628 XE Delft, Netherlands
Department of Human Genetics, Leiden University Medical Center, Einthovenweg 0, 333 ZC Leiden, Netherlands
-
SIB Swiss Institute of Bioinformatics · University of Zurich · Allen Institute for Neural Dynamics
Affiliation as printed
Allen Institute for Neural Dynamics, Seattle, WA, USA
Department of Molecular Life Sciences and SIB Swiss Institute of Bioinformatics, University of Zurich, Winterthurerstrasse 190, 8057 Zurich, Switzerland
-
Agency for Science, Technology and Research · Berlin Institute of Health at Charité - Universitätsmedizin Berlin · Genome Institute of Singapore
Affiliation as printed
Genome Institute of Singapore, Agency for Science, Technology and Research (A*STAR), Singapore, Singapore
Berlin Institute of Health at Charité – Universitätsmedizin Berlin, Center of Digital Health, Charitéplatz 1, 10117 Berlin, Germany
-
Centre Hospitalier Universitaire Vaudois
Affiliation as printed
Biomedical Data Science Center, Centre Hospitalier Universitaire Vaudois, Lausanne, Switzerland
-
Affiliation as printed
Nature Genetics, Berlin, Germany
-
Centre for Research and Technology Hellas · Berlin Institute of Health at Charité - Universitätsmedizin Berlin
Affiliation as printed
Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Center of Digital, Berlin, Germany
Institute of Applied Biosciences, Centre for Research and Technology Hellas, Thessaloniki, Greece
-
SIB Swiss Institute of Bioinformatics · University of Basel
Affiliation as printed
Biozentrum, University of Basel, Basel, Switzerland
SIB Swiss Institute of Bioinformatics, Quartier Sorge Batiment Amphipole, Lausanne, Switzerland
-
Affiliation as printed
German Cancer Research Center (DKFZ), Division of Computational Genomics and Systems Genetics, Heidelberg, Germany
The German Human Genome-Phenome Archive, Heidelberg, Germany
-
Helmholtz Munich · Institute of Computational Biology
Affiliation as printed
Helmholtz Zentrum München, Institute of Computational Biology, Neuherberg, Germany
-
Swiss Center for Electronics and Microtechnology (Switzerland) · ETH Zurich
Affiliation as printed
CSEM, Allschwil, Switzerland
ETHZ, BMIC - Biomedical Image Computing lab, Zurich, Switzerland
-
Delft University of Technology
Affiliation as printed
Delft Bioinformatics Lab, Delft University of Technology, Delft, the Netherlands
-
University of Copenhagen · LEO Foundation · Technical University of Denmark
Affiliation as printed
DTU Health Tech, Technical University of Denmark, Ørsteds Plads, Kongens Lyngby, Denmark
LEO Foundation Skin Immunology Research Center, Department of Immunology and Microbiology, University of Copenhagen, Copenhagen, Denmark
-
Affiliation as printed
OMAPiX GmbH Raiffeisenstraße 3, Langenfeld, Germany
-
Technical University of Munich
Affiliation as printed
Technical University of Munich, School of Medicine, Institute of Virology, Munich, Germany
-
Affiliation as printed
University of Michigan, Undergraduate Research Opportunity Program, Ann Arbor, MI, USA
-
Czech Academy of Sciences · Czech Academy of Sciences, Institute of Molecular Genetics · University of Chemistry and Technology, Prague
Affiliation as printed
Department of Informatics and Chemistry, Faculty of Chemical Technology, University of Chemistry and Technology, Prague, Czech Republic
Laboratory of Genomics and Bioinformatics, Institute of Molecular Genetics of the Czech Academy of Sciences, Prague, Czech Republic
-
Universitätsklinikum Aachen · RWTH Aachen University
Affiliation as printed
Department of Nephrology, Rheumatology, and Clinical Immunology, University Hospital RWTH Aachen, Aachen, Germany
-
Agency for Science, Technology and Research · Genome Institute of Singapore
Affiliation as printed
Genome Institute of Singapore, Agency for Science, Technology and Research (A*STAR), Singapore, Singapore
-
Science for Life Laboratory · National Bioinformatics Infrastructure Sweden · Chalmers University of Technology
Affiliation as printed
Department of Biology and Biological Engineering, Chalmers University of Technology, Gothenburg, Sweden
National Bioinformatics Infrastructure of Sweden, Scilifelab, Sweden
-
Affiliation as printed
Division of Artificial Intelligence in Oncology (B450), German Cancer Research Center (DKFZ), Heidelberg, Germany
-
Agency for Science, Technology and Research · Genome Institute of Singapore
Affiliation as printed
Genome Institute of Singapore, Agency for Science, Technology and Research (A*STAR), Singapore, Singapore
-
Affiliation as printed
Data Science Lab, Hertie School, Berlin, Germany
-
Affiliation as printed
Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland
-
Berlin Institute of Health at Charité - Universitätsmedizin Berlin
Affiliation as printed
Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Center of Digital, Berlin, Germany
-
SIB Swiss Institute of Bioinformatics · Swiss Cancer Center Léman · University of Lausanne
Affiliation as printed
Department of Computational Biology, University of Lausanne, Lausanne, Switzerland
SIB Swiss Institute of Bioinformatics, Quartier Sorge Batiment Amphipole, Lausanne, Switzerland
Swiss Cancer Center Léman, Lausanne, Switzerland
-
German Cancer Research Center · Heidelberg University · German Center for Lung Research · University Medical Centre Mannheim · Medizinische Fakultät Mannheim
Affiliation as printed
DKFZ Hector Cancer Institute at the University Medical Center Mannheim, Mannheim, Germany
Department of Personalized Oncology, University Hospital Mannheim, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany
Division Applied Bioinformatics (B330), German Cancer Research Center (DKFZ), Heidelberg, Germany
Division of Personalized Medical Oncology (A420), German Cancer Research Center (DKFZ), Heidelberg, Germany
German Center for Lung Research (DZL), Giessen, Germany
-
SIB Swiss Institute of Bioinformatics
Affiliation as printed
SIB Swiss Institute of Bioinformatics, Quartier Sorge Batiment Amphipole, Lausanne, Switzerland
-
European Bioinformatics Institute
Affiliation as printed
European Molecular Biology Laboratory, European Bioinformatics Institute, Hinxton, UK
-
Berlin Institute of Health at Charité - Universitätsmedizin Berlin
Affiliation as printed
Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Center of Digital, Berlin, Germany
-
Delft University of Technology
Affiliation as printed
Delft Bioinformatics Lab, Delft University of Technology, Delft, the Netherlands
-
Centre Hospitalier Universitaire Vaudois
Affiliation as printed
Biomedical Data Science Center, Centre Hospitalier Universitaire Vaudois, Lausanne, Switzerland
-
Universitätsklinikum Aachen · RWTH Aachen University
Affiliation as printed
Department of Nephrology, Rheumatology, and Clinical Immunology, University Hospital RWTH Aachen, Aachen, Germany
-
Brian Long corresponding
Allen Institute for Brain Science
Affiliation as printed
Allen Institute for Brain Science, Seattle, WA, USA. brianl@alleninstitute.org
-
Leiden University Medical Center · Delft University of Technology
Affiliation as printed
Delft Bioinformatics Lab, Delft University of Technology, Delft, the Netherlands. a.mahfouz@lumc.nl
Department of Human Genetics, Leiden University Medical Center, Leiden, the Netherlands. a.mahfouz@lumc.nl
-
Mark D. Robinson corresponding
SIB Swiss Institute of Bioinformatics · University of Zurich
Affiliation as printed
Department of Molecular Life Sciences and SIB Swiss Institute of Bioinformatics, University of Zurich, Zurich, Switzerland. mark.robinson@mls.uzh.ch
-
Naveed Ishaque corresponding
Berlin Institute of Health at Charité - Universitätsmedizin Berlin
Affiliation as printed
Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Center of Digital, Berlin, Germany. naveed.ishaque@bih-charite.de
Cited by 4 stored of 4
4 results
No patents citing this paper on Lens.org (checked 2026-10-06).