Covariate-specific ROC curve analysis can accommodate differences between covariate subgroups in the evaluation of diagnostic accuracy
Journal of Clinical Epidemiology, vol. 160, pp. 14–23
Abstract
OBJECTIVES: We present an illustrative application of methods that account for covariates in receiver operating characteristic (ROC) curve analysis, using individual patient data on D-dimer testing for excluding pulmonary embolism. STUDY DESIGN AND SETTING: Bayesian nonparametric covariate-specific ROC curves were constructed to examine the performance/positivity thresholds in covariate subgroups. Standard ROC curves were constructed. Three scenarios were outlined based on comparison between subgroups and standard ROC curve conclusion: (1) identical distribution/identical performance, (2) different distribution/identical performance, and (3) different distribution/different performance. Scenarios were illustrated using clinical covariates. Covariate-adjusted ROC curves were also constructed. RESULTS: Age groups had prominent differences in D-dimer concentration, paired with differences in performance (Scenario 3). Different positivity thresholds were required to achieve the same level of sensitivity. D-dimer had identical performance, but different distributions for YEARS algorithm items (Scenario 2), and similar distributions for sex (Scenario 1). For the later covariates, comparable positivity thresholds achieved the same sensitivity. All covariate-adjusted models had AUCs comparable to the standard approach. CONCLUSION: Subgroup differences in performance and distribution of results can indicate that the conventional ROC curve is not a fair representation of test performance. Estimating conditional ROC curves can improve the ability to select thresholds with greater applicability.
Authors 7
-
Jenny Lee corresponding
Amsterdam UMC Location University of Amsterdam
Affiliation as printed
Epidemiology and Data Science, Amsterdam UMC location University of Amsterdam, Amsterdam, The Netherlands. Electronic address: j.a.lee@amsterdamumc.nl
Epidemiology and Data Science, Amsterdam UMC location University of Amsterdam, Amsterdam, The Netherlands
-
Amsterdam UMC Location University of Amsterdam · Amsterdam University Medical Centers · Amsterdam Cardiovascular Sciences
Affiliation as printed
Department of Vascular Medicine, Amsterdam UMC, location University of Amsterdam, Amsterdam, The Netherlands; Amsterdam Cardiovascular Sciences, Pulmonary Hypertension & Thrombosis, Amsterdam, The Netherlands
Amsterdam Cardiovascular Sciences, Pulmonary Hypertension & Thrombosis, Amsterdam, The Netherlands
Department of Vascular Medicine, Amsterdam UMC, location University of Amsterdam, Amsterdam, The Netherlands
-
Fukushima Medical University · Utrecht University · University Medical Center Utrecht
Affiliation as printed
Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands; Department of General Medicine, Shirakawa Satellite for Teaching And Research (STAR), Fukushima Medical University, 2-1 Toyochi Kamiyajiro, Shirakawa, Fukushima, 961-0005, Japan
Department of General Medicine, Shirakawa Satellite for Teaching And Research (STAR), Fukushima Medical University, 2-1 Toyochi Kamiyajiro, Shirakawa, Fukushima, 961-0005, Japan
Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
-
Leiden University · Leiden University Medical Center
Affiliation as printed
Department of Medicine - Thrombosis and Hemostasis, Leiden University Medical Center, Leiden University, Leiden, The Netherlands
-
Utrecht University · University Medical Center Utrecht
Affiliation as printed
Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
-
Fukushima Medical University · University of Virginia
Affiliation as printed
Department of General Medicine, Shirakawa Satellite for Teaching And Research (STAR), Fukushima Medical University, 2-1 Toyochi Kamiyajiro, Shirakawa, Fukushima, 961-0005, Japan; Department of Data Science, University of Virginia, Charlottesville, VA, USA
Department of Data Science, University of Virginia, Charlottesville, VA, USA
Department of General Medicine, Shirakawa Satellite for Teaching And Research (STAR), Fukushima Medical University, 2-1 Toyochi Kamiyajiro, Shirakawa, Fukushima, 961-0005, Japan
-
Amsterdam UMC Location University of Amsterdam
Affiliation as printed
Epidemiology and Data Science, Amsterdam UMC location University of Amsterdam, Amsterdam, The Netherlands
Cited by 24 stored of 25
No patents citing this paper on Lens.org (checked 2026-10-11).