Securing Collaborative Medical AI by Using Differential Privacy: Domain Transfer for Classification of Chest Radiographs

Radiology Artificial Intelligence, vol. 6

Abstract

Purpose To investigate the integration of differential privacy (DP) and analyze its impact on model performance as compared with models trained without DP. Materials and Methods Leveraging more than 590 000 chest radiographs from five institutions, including VinDr-CXR from Vietnam, ChestX-ray14 and CheXpert from the United States, UKA-CXR from Germany, and PadChest from Spain, the authors evaluated the efficacy of DP-enhanced domain transfer (DP-DT) in classifying cardiomegaly, pleural effusion, pneumonia, atelectasis, and healthy individuals. Diagnostic performance and sex-specific and age-specific demographic fairness of DP-DT and of non–DP-DT models were compared using the area under the receiver operating characteristic curve (AUC) as the main metric, as well as accuracy, sensitivity, and specificity as secondary metrics, and evaluated for statistical significance using paired Student t tests. Results Even with high privacy levels (ε ≈ 1), DP-DT showed no evidence of differences compared with non–DP-DT in terms of a decrease in AUC of cross-institutional performance as compared with single-institutional performance (VinDr-CXR: 0.07 vs 0.07, P = .96; ChestX-ray14: 0.07 vs 0.06, P = .12; CheXpert: 0.07 vs 0.07, P = .18; UKA-CXR: 0.18 vs 0.18, P = .90; and PadChest: 0.07 vs 0.07, P = .35). Furthermore, AUC differences between DP-DT and non–DP-DT models were less than 1% for all sex subgroups (P > .33 for female and P > .22 for male, for all domains) and nearly all age subgroups (P > .16 for younger participants, P > .33 for adults, and P > .27 for older adults, for nearly all domains). Conclusion Cross-institutional performance of artificial intelligence models was not affected by DP. Keywords: Convolutional Neural Network (CNN), Transfer Learning, Supervised Learning, Diagnosis, Forensics, Computer Applications–General, Image Postprocessing, Informatics, Neural Networks, Thorax, Computer-Aided Diagnosis, Deep Learning, Domain Transfer, Differential Privacy, Privacy-Preserving AI, Chest Radiograph Supplemental material is available for this article. © RSNA, 2023 See also the commentary by Suri and Summers in this issue.

Authors 9

1. Soroosh Tayebi Arasteh Aachen Department of Diagnostic and Interventional Radiology

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Pauwelsstr 30, 52074 Aachen, Germany

2. Mahshad Lotfinia Aachen Institute of Heat and Mass Transfer

   RWTH Aachen University

   Affiliation as printed

   Institute of Heat and Mass Transfer, RWTH Aachen University, Aachen, Germany

3. Teresa Nolte Aachen Department of Diagnostic and Interventional Radiology

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Pauwelsstr 30, 52074 Aachen, Germany

4. Marwin-Jonathan Sähn Aachen Department of Diagnostic and Interventional Radiology

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Pauwelsstr 30, 52074 Aachen, Germany

5. Peter Isfort Aachen Department of Diagnostic and Interventional Radiology

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Pauwelsstr 30, 52074 Aachen, Germany

6. Christiane Katharina Kuhl Aachen Department of Diagnostic and Interventional Radiology

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Pauwelsstr 30, 52074 Aachen, Germany

7. Sven Nebelung Aachen Department of Diagnostic and Interventional Radiology

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Pauwelsstr 30, 52074 Aachen, Germany

8. Georgios A. Kaissis

   Helmholtz Munich · Technical University of Munich

   Affiliation as printed

   Institute for Machine Learning in Biomedical Imaging, Helmholtz-Zentrum Munich, Neuherberg, Germany

   Institute of Diagnostic and Interventional Radiology and Department of Artificial Intelligence in Healthcare and Medicine, Technical University of Munich, Munich, Germany

9. Daniel Truhn Aachen Department of Diagnostic and Interventional Radiology

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Pauwelsstr 30, 52074 Aachen, Germany