Predicting Depression Risk in Patients With Cancer Using Multimodal Data: Algorithm Development Study

JMIR Medical Informatics, vol. 12, pp. e51925

Abstract

BACKGROUND: Patients with cancer starting systemic treatment programs, such as chemotherapy, often develop depression. A prediction model may assist physicians and health care workers in the early identification of these vulnerable patients. OBJECTIVE: This study aimed to develop a prediction model for depression risk within the first month of cancer treatment. METHODS: We included 16,159 patients diagnosed with cancer starting chemo- or radiotherapy treatment between 2008 and 2021. Machine learning models (eg, least absolute shrinkage and selection operator [LASSO] logistic regression) and natural language processing models (Bidirectional Encoder Representations from Transformers [BERT]) were used to develop multimodal prediction models using both electronic health record data and unstructured text (patient emails and clinician notes). Model performance was assessed in an independent test set (n=5387, 33%) using area under the receiver operating characteristic curve (AUROC), calibration curves, and decision curve analysis to assess initial clinical impact use. RESULTS: Among 16,159 patients, 437 (2.7%) received a depression diagnosis within the first month of treatment. The LASSO logistic regression models based on the structured data (AUROC 0.74, 95% CI 0.71-0.78) and structured data with email classification scores (AUROC 0.74, 95% CI 0.71-0.78) had the best discriminative performance. The BERT models based on clinician notes and structured data with email classification scores had AUROCs around 0.71. The logistic regression model based on email classification scores alone performed poorly (AUROC 0.54, 95% CI 0.52-0.56), and the model based solely on clinician notes had the worst performance (AUROC 0.50, 95% CI 0.49-0.52). Calibration was good for the logistic regression models, whereas the BERT models produced overly extreme risk estimates even after recalibration. There was a small range of decision thresholds for which the best-performing model showed promising clinical effectiveness use. The risks were underestimated for female and Black patients. CONCLUSIONS: The results demonstrated the potential and limitations of machine learning and multimodal models for predicting depression risk in patients with cancer. Future research is needed to further validate these models, refine the outcome label and predictors related to mental health, and address biases across subgroups.

Authors 8

1. Anne de Hond Aachen Clinical AI Implementation and Research Lab Department of Biomedical Data Sciences

   Leiden University · Leiden University Medical Center · Stanford Medicine · Stanford University

   Affiliation as printed

   Clinical AI Implementation and Research Lab, Leiden University Medical Centre, Leiden, Netherlands

   Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, Netherlands

   Department of Medicine (Biomedical Informatics), Stanford Medicine, Stanford University, Stanford, CA, United States

2. Marieke Meija van Buchem Aachen Clinical AI Implementation and Research Lab Department of Biomedical Data Sciences

   Leiden University · Leiden University Medical Center · Stanford Medicine · Stanford University

   Affiliation as printed

   Clinical AI Implementation and Research Lab, Leiden University Medical Centre, Leiden, Netherlands

   Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, Netherlands

   Department of Medicine (Biomedical Informatics), Stanford Medicine, Stanford University, Stanford, CA, United States

3. Claudio Fanconi

   ETH Zurich · Stanford Medicine · Stanford University

   Affiliation as printed

   Department of Electrical Engineering and Information Technology, ETH Zürich, Zürich, Switzerland

   Department of Medicine (Biomedical Informatics), Stanford Medicine, Stanford University, Stanford, CA, United States

4. Mohana Roy

   Stanford Medicine · Stanford University

   Affiliation as printed

   Department of Medical Oncology, Stanford Medicine, Stanford University, Stanford, CA, United States

5. Douglas W. Blayney

   Stanford Medicine · Stanford University

   Affiliation as printed

   Department of Medical Oncology, Stanford Medicine, Stanford University, Stanford, CA, United States

6. Ilse M.J. Kant Aachen Clinical AI Implementation and Research Lab

   Leiden University · Leiden University Medical Center · University Medical Center Utrecht

   Affiliation as printed

   Clinical AI Implementation and Research Lab, Leiden University Medical Centre, Leiden, Netherlands

   Department of Digital Health, University Medical Centre Utrecht, Utrecht, Netherlands

7. Ewout Willem Steyerberg Aachen Clinical AI Implementation and Research Lab Department of Biomedical Data Sciences

   Leiden University · Leiden University Medical Center

   Affiliation as printed

   Clinical AI Implementation and Research Lab, Leiden University Medical Centre, Leiden, Netherlands

   Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, Netherlands

8. Tina Hernandez‐Boussard

   Stanford Medicine · Stanford University

   Affiliation as printed

   Department of Biomedical Data Science, Stanford University, Stanford, CA, United States

   Department of Epidemiology & Population Health (by courtesy), Stanford University, Stanford, CA, United States

   Department of Medicine (Biomedical Informatics), Stanford Medicine, Stanford University, Stanford, CA, United States