A realistic benchmark for differential abundance testing and confounder adjustment in human microbiome studies
Genome biology, vol. 25, pp. 247
Abstract
BACKGROUND: In microbiome disease association studies, it is a fundamental task to test which microbes differ in their abundance between groups. Yet, consensus on suitable or optimal statistical methods for differential abundance testing is lacking, and it remains unexplored how these cope with confounding. Previous differential abundance benchmarks relying on simulated datasets did not quantitatively evaluate the similarity to real data, which undermines their recommendations. RESULTS: Our simulation framework implants calibrated signals into real taxonomic profiles, including signals mimicking confounders. Using several whole meta-genome and 16S rRNA gene amplicon datasets, we validate that our simulated data resembles real data from disease association studies much more than in previous benchmarks. With extensively parametrized simulations, we benchmark the performance of nineteen differential abundance methods and further evaluate the best ones on confounded simulations. Only classic statistical methods (linear models, the Wilcoxon test, t-test), limma, and fastANCOM properly control false discoveries at relatively high sensitivity. When additionally considering confounders, these issues are exacerbated, but we find that adjusted differential abundance testing can effectively mitigate them. In a large cardiometabolic disease dataset, we showcase that failure to account for covariates such as medication causes spurious association in real-world applications. CONCLUSIONS: Tight error control is critical for microbiome association studies. The unsatisfactory performance of many differential abundance methods and the persistent danger of unchecked confounding suggest these contribute to a lack of reproducibility among such studies. We have open-sourced our simulation and benchmarking software to foster a much-needed consolidation of statistical methodology for microbiome research.
Authors 4
-
European Molecular Biology Laboratory · European Molecular Biology Laboratory
Affiliation as printed
Structural and Computational Biology Unit (SCB), European Molecular Biology Laboratory (EMBL), Heidelberg, Germany
-
Max Delbrück Center · Humboldt-Universität zu Berlin · Charité - Universitätsmedizin Berlin
Affiliation as printed
Charité-Universitätsmedizin Berlin (a corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin), Berlin, Germany
Experimental and Clinical Research Center (ECRC), a cooperation of the Max-Delbrück Center and Charité-Universitätsmedizin, Berlin, Germany
Max-Delbrück Center for Molecular Medicine in the Helmholtz Association (MDC), Berlin, Germany
-
Sofia Kirke Forslund corresponding
Max Delbrück Center · Humboldt-Universität zu Berlin · German Centre for Cardiovascular Research · European Molecular Biology Laboratory · European Molecular Biology Laboratory · Charité - Universitätsmedizin Berlin
Affiliation as printed
Charité-Universitätsmedizin Berlin (a corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin), Berlin, Germany. sofia.forslund@mdc-berlin.de
Experimental and Clinical Research Center (ECRC), a cooperation of the Max-Delbrück Center and Charité-Universitätsmedizin, Berlin, Germany. sofia.forslund@mdc-berlin.de
German Center for Cardiovascular Research (DZHK), Partner Site Berlin, Berlin, Germany. sofia.forslund@mdc-berlin.de
Max-Delbrück Center for Molecular Medicine in the Helmholtz Association (MDC), Berlin, Germany. sofia.forslund@mdc-berlin.de
Structural and Computational Biology Unit (SCB), European Molecular Biology Laboratory (EMBL), Heidelberg, Germany. sofia.forslund@mdc-berlin.de
-
Georg Zeller corresponding Aachen Centre for Infectious Diseases Center for Microbiome Analyses and Therapeutics
Leiden University · Leiden University Medical Center · European Molecular Biology Laboratory · European Molecular Biology Laboratory
Affiliation as printed
Center for Infectious Diseases (LUCID), Leiden University, Leiden University Medical Center (LUMC), Leiden, Netherlands. georg.zeller@gmail.com
Center for Microbiome Analyses and Therapeutics (CMAT), Leiden University Medical Center, Leiden, Netherlands. georg.zeller@gmail.com
Structural and Computational Biology Unit (SCB), European Molecular Biology Laboratory (EMBL), Heidelberg, Germany. georg.zeller@gmail.com
Cited by 45 stored of 44
No patents citing this paper on Lens.org (checked 2026-10-11).