Performance of Five Metagenomic Classifiers for Virus Pathogen Detection Using Respiratory Samples from a Clinical Cohort

Pathogens, vol. 11, pp. 340

Abstract

Viral metagenomics is increasingly applied in clinical diagnostic settings for detection of pathogenic viruses. While several benchmarking studies have been published on the use of metagenomic classifiers for abundance and diversity profiling of bacterial populations, studies on the comparative performance of the classifiers for virus pathogen detection are scarce. In this study, metagenomic data sets (n = 88) from a clinical cohort of patients with respiratory complaints were used for comparison of the performance of five taxonomic classifiers: Centrifuge, Clark, Kaiju, Kraken2, and Genome Detective. A total of 1144 positive and negative PCR results for a total of 13 respiratory viruses were used as gold standard. Sensitivity and specificity of these classifiers ranged from 83 to 100% and 90 to 99%, respectively, and was dependent on the classification level and data pre-processing. Exclusion of human reads generally resulted in increased specificity. Normalization of read counts for genome length resulted in a minor effect on overall performance, however it negatively affected the detection of targets with read counts around detection level. Correlation of sequence read counts with PCR Ct-values varied per classifier, data pre-processing (R2 range 15.1–63.4%), and per virus, with outliers up to 3 log10 reads magnitude beyond the predicted read count for viruses with high sequence diversity. In this benchmarking study, sensitivity and specificity were within the ranges of use for diagnostic practice when the cut-off for defining a positive result was considered per classifier.

Authors 10

1. Ellen C. Carbo corresponding Aachen Department of Medical Microbiology

   Leiden University Medical Center

   Affiliation as printed

   Department of Medical Microbiology, Leiden University Medical Center, 2333 ZA Leiden, The Netherlands

2. Igor A. Sidorov Aachen Department of Medical Microbiology

   Leiden University Medical Center

   Affiliation as printed

   Department of Medical Microbiology, Leiden University Medical Center, 2333 ZA Leiden, The Netherlands

3. Anneloes L. van Rijn-Klink Aachen Department of Medical Microbiology

   Leiden University Medical Center

   Affiliation as printed

   Department of Medical Microbiology, Leiden University Medical Center, 2333 ZA Leiden, The Netherlands

4. Sander van Boheemen Aachen Department of Medical Microbiology

   Leiden University Medical Center · Erasmus MC

   Affiliation as printed

   Department of Medical Microbiology, Leiden University Medical Center, 2333 ZA Leiden, The Netherlands

   Department of Viroscience, Erasmus Medical Center, 3015 GD Rotterdam, The Netherlands

5. Hailiang Mei Aachen Sequencing Analysis Support Core Department of Biomedical Data Sciences

   Leiden University Medical Center

   Affiliation as printed

   Sequencing Analysis Support Core, Department of Biomedical Data Sciences, Leiden University Medical Center, 2333 ZA Leiden, The Netherlands

6. Pieter Sicco Hiemstra Aachen Dept of Pulmonology, Leiden University Medical Center

   Leiden University Medical Center

   Affiliation as printed

   Department of Pulmonology, Leiden University Medical Center, 2333 ZA Leiden, The Netherlands

7. Tomas Eagan

   Haukeland University Hospital

   Affiliation as printed

   Department of Thoracic Medicine, Haukeland University Hospital, 5021 Bergen, Norway

8. Eric Claas Aachen Department of Medical Microbiology

   Leiden University Medical Center

   Affiliation as printed

   Department of Medical Microbiology, Leiden University Medical Center, 2333 ZA Leiden, The Netherlands

9. Aloys C.M. Kroes Aachen Department of Medical Microbiology

   Leiden University Medical Center

   Affiliation as printed

   Department of Medical Microbiology, Leiden University Medical Center, 2333 ZA Leiden, The Netherlands

10. Jutte J.C. de Vries Aachen Department of Medical Microbiology

    Leiden University Medical Center

    Affiliation as printed

    Department of Medical Microbiology, Leiden University Medical Center, 2333 ZA Leiden, The Netherlands