Applying natural language processing to patient messages to identify depression concerns in cancer patients

Journal of the American Medical Informatics Association, vol. 31, pp. 2255–2262

Abstract

OBJECTIVE: This study aims to explore and develop tools for early identification of depression concerns among cancer patients by leveraging the novel data source of messages sent through a secure patient portal. MATERIALS AND METHODS: We developed classifiers based on logistic regression (LR), support vector machines (SVMs), and 2 Bidirectional Encoder Representations from Transformers (BERT) models (original and Reddit-pretrained) on 6600 patient messages from a cancer center (2009-2022), annotated by a panel of healthcare professionals. Performance was compared using AUROC scores, and model fairness and explainability were examined. We also examined correlations between model predictions and depression diagnosis and treatment. RESULTS: BERT and RedditBERT attained AUROC scores of 0.88 and 0.86, respectively, compared to 0.79 for LR and 0.83 for SVM. BERT showed bigger differences in performance across sex, race, and ethnicity than RedditBERT. Patients who sent messages classified as concerning had a higher chance of receiving a depression diagnosis, a prescription for antidepressants, or a referral to the psycho-oncologist. Explanations from BERT and RedditBERT differed, with no clear preference from annotators. DISCUSSION: We show the potential of BERT and RedditBERT in identifying depression concerns in messages from cancer patients. Performance disparities across demographic groups highlight the need for careful consideration of potential biases. Further research is needed to address biases, evaluate real-world impacts, and ensure responsible integration into clinical settings. CONCLUSION: This work represents a significant methodological advancement in the early identification of depression concerns among cancer patients. Our work contributes to a route to reduce clinical burden while enhancing overall patient care, leveraging BERT-based models.

Authors 8

1. Marieke Meija van Buchem corresponding Aachen Clinical Artificial Intelligence Implementation and Research Lab (CAIRELab)

   Leiden University Medical Center · Stanford Medicine · Stanford University

   Affiliation as printed

   Clinical Artificial Intelligence Implementation and Research Lab (CAIRELab), Leiden University Medical Center , Leiden 2333ZN, The Netherlands

   Department of Medicine (Biomedical Informatics), Stanford University , Stanford, CA 94305, United States

2. Anne de Hond

   University Medical Center Utrecht · Stanford Medicine · Stanford University

   Affiliation as printed

   Department of Medicine (Biomedical Informatics), Stanford University , Stanford, CA 94305, United States

   Julius Centre for Health Sciences and Primary Care, University Medical Center , Utrecht 3584CX, The Netherlands

3. Claudio Fanconi

   ETH Zurich · Stanford Medicine · Stanford University

   Affiliation as printed

   Department of Information Technology and Electrical Engineering, ETH Zürich , Zürich 8092, Switzerland

   Department of Medicine (Biomedical Informatics), Stanford University , Stanford, CA 94305, United States

4. Vaibhavi B. Shah

   Stanford Medicine · Stanford University

   Affiliation as printed

   Department of Medicine (Biomedical Informatics), Stanford University , Stanford, CA 94305, United States

5. Max Schuessler

   Stanford University

   Affiliation as printed

   Department of Biomedical Data Science, Stanford University , Stanford, CA 94305, United States

6. Ilse M.J. Kant

   University Medical Center Utrecht

   Affiliation as printed

   Department of Digital Health, University Medical Center Utrecht , Utrecht 3584CX, The Netherlands

7. Ewout Willem Steyerberg Aachen Clinical Artificial Intelligence Implementation and Research Lab (CAIRELab) Department of Biomedical Data Sciences

   Leiden University Medical Center

   Affiliation as printed

   Clinical Artificial Intelligence Implementation and Research Lab (CAIRELab), Leiden University Medical Center , Leiden 2333ZN, The Netherlands

   Department of Biomedical Data Sciences, Leiden University Medical Center , Leiden 2333ZN, The Netherlands

8. Tina Hernandez‐Boussard

   Stanford Medicine · Stanford University

   Affiliation as printed

   Department of Biomedical Data Science, Stanford University , Stanford, CA 94305, United States

   Department of Medicine (Biomedical Informatics), Stanford University , Stanford, CA 94305, United States