Estimating individualized treatment effects from randomized controlled trials: a simulation study to compare risk-based approaches

BMC Medical Research Methodology, vol. 23, pp. 74

Abstract

BACKGROUND: Baseline outcome risk can be an important determinant of absolute treatment benefit and has been used in guidelines for "personalizing" medical decisions. We compared easily applicable risk-based methods for optimal prediction of individualized treatment effects. METHODS: We simulated RCT data using diverse assumptions for the average treatment effect, a baseline prognostic index of risk, the shape of its interaction with treatment (none, linear, quadratic or non-monotonic), and the magnitude of treatment-related harms (none or constant independent of the prognostic index). We predicted absolute benefit using: models with a constant relative treatment effect; stratification in quarters of the prognostic index; models including a linear interaction of treatment with the prognostic index; models including an interaction of treatment with a restricted cubic spline transformation of the prognostic index; an adaptive approach using Akaike's Information Criterion. We evaluated predictive performance using root mean squared error and measures of discrimination and calibration for benefit. RESULTS: The linear-interaction model displayed optimal or close-to-optimal performance across many simulation scenarios with moderate sample size (N = 4,250; ~ 785 events). The restricted cubic splines model was optimal for strong non-linear deviations from a constant treatment effect, particularly when sample size was larger (N = 17,000). The adaptive approach also required larger sample sizes. These findings were illustrated in the GUSTO-I trial. CONCLUSIONS: An interaction between baseline risk and treatment assignment should be considered to improve treatment effect predictions.

Authors 5

1. Alexandros Rekkas corresponding

   Erasmus MC

   Affiliation as printed

   Department of Medical Informatics, Erasmus Medical Center, P.O. Box 2040, 3000, CA, Rotterdam, The Netherlands. a.rekkas@erasmusmc.nl

   Department of Medical Informatics, Erasmus Medical Center, P.O. Box 2040, 3000, CA, Rotterdam, The Netherlands

2. Peter R. Rijnbeek

   Erasmus MC

   Affiliation as printed

   Department of Medical Informatics, Erasmus Medical Center, P.O. Box 2040, 3000, CA, Rotterdam, The Netherlands

3. David M. Kent

   Tufts Medical Center

   Affiliation as printed

   Predictive Analytics and Comparative Effectiveness Center, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, Boston, MA, USA

4. Ewout Willem Steyerberg Aachen Department of Biomedical Data Sciences

   Leiden University Medical Center

   Affiliation as printed

   Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands

5. David V. Klaveren

   Erasmus MC

   Affiliation as printed

   Department of Public Health, Erasmus Medical Center, Rotterdam, The Netherlands