Combining CNN-based histologic whole slide image analysis and patient data to improve skin cancer classification
European Journal of Cancer, vol. 149, pp. 94–101
Abstract
BACKGROUND: Clinicians and pathologists traditionally use patient data in addition to clinical examination to support their diagnoses. OBJECTIVES: We investigated whether a combination of histologic whole slides image (WSI) analysis based on convolutional neural networks (CNNs) and commonly available patient data (age, sex and anatomical site of the lesion) in a binary melanoma/nevus classification task could increase the performance compared with CNNs alone. METHODS: We used 431 WSIs from two different laboratories and analysed the performance of classifiers that used the image or patient data individually or three common fusion techniques. Furthermore, we tested a naive combination of patient data and an image classifier: for cases interpreted as 'uncertain' (CNN output score <0.7), the decision of the CNN was replaced by the decision of the patient data classifier. RESULTS: The CNN on its own achieved the best performance (mean ± standard deviation of five individual runs) with AUROC of 92.30% ± 0.23% and balanced accuracy of 83.17% ± 0.38%. While the classification performance was not significantly improved in general by any of the tested fusions, naive strategy of replacing the image classifier with the patient data classifier on slides with low output scores improved balanced accuracy to 86.72% ± 0.36%. CONCLUSION: In most cases, the CNN on its own was so accurate that patient data integration did not provide any benefit. However, incorporating patient data for lesions that were classified by the CNN with low 'confidence' improved balanced accuracy.
Authors 31
-
German Cancer Research Center · National Center for Tumor Diseases
Affiliation as printed
Digital Biomarkers for Oncology Group, National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
-
German Cancer Research Center · National Center for Tumor Diseases
Affiliation as printed
Digital Biomarkers for Oncology Group, National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
-
German Cancer Research Center · National Center for Tumor Diseases
Affiliation as printed
Digital Biomarkers for Oncology Group, National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
-
Berlin Institute of Health at Charité - Universitätsmedizin Berlin · Charité - Universitätsmedizin Berlin
Affiliation as printed
Department of Clinical-Translational Sciences, Charité University Medicine and Berlin Institute of Health (BIH), Berlin, Germany
-
German Cancer Research Center · Heidelberg University
Affiliation as printed
Department of Dermatology, Heidelberg University, Mannheim, Germany; Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany
Department of Dermatology, Heidelberg University, Mannheim, Germany
Skin Cancer Unit, German Cancer Research Center (DKFZ), Heidelberg, Germany
-
University Hospital Carl Gustav Carus · Nationales Centrum für Tumorerkrankungen Dresden · Technische Universität Dresden
Affiliation as printed
Skin Cancer Center at the University Cancer Centre and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität, Dresden, Germany
-
University Hospital Carl Gustav Carus · Nationales Centrum für Tumorerkrankungen Dresden · Technische Universität Dresden
Affiliation as printed
Skin Cancer Center at the University Cancer Centre and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität, Dresden, Germany
-
University Hospital Carl Gustav Carus · Nationales Centrum für Tumorerkrankungen Dresden · Technische Universität Dresden
Affiliation as printed
Skin Cancer Center at the University Cancer Centre and National Center for Tumor Diseases Dresden, Department of Dermatology, University Hospital Carl Gustav Carus, Technische Universität, Dresden, Germany
-
University Hospital Schleswig-Holstein
Affiliation as printed
Department of Dermatology, University Hospital of Kiel, Germany
-
Ludwig-Maximilians-Universität München
Affiliation as printed
Department of Dermatology and Allergology, Ludwig Maximilian University of Munich, Munich, Germany
-
Ludwig-Maximilians-Universität München
Affiliation as printed
Department of Dermatology and Allergology, Ludwig Maximilian University of Munich, Munich, Germany
-
Ludwig-Maximilians-Universität München
Affiliation as printed
Department of Dermatology and Allergology, Ludwig Maximilian University of Munich, Munich, Germany
-
Charité - Universitätsmedizin Berlin
Affiliation as printed
Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Berlin, Germany
Department of Dermatology, Venereology and Allergology, Charité – Universitätsmedizin Berlin, Berlin, Germany
-
Charité - Universitätsmedizin Berlin
Affiliation as printed
Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Berlin, Germany
Department of Dermatology, Venereology and Allergology, Charité – Universitätsmedizin Berlin, Berlin, Germany
-
Charité - Universitätsmedizin Berlin
Affiliation as printed
Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Berlin, Germany
Department of Dermatology, Venereology and Allergology, Charité – Universitätsmedizin Berlin, Berlin, Germany
-
Charité - Universitätsmedizin Berlin
Affiliation as printed
Department of Dermatology, Venereology and Allergology, Charité - Universitätsmedizin Berlin, Berlin, Germany
Department of Dermatology, Venereology and Allergology, Charité – Universitätsmedizin Berlin, Berlin, Germany
-
Dermatopathologie Friedrichshafen
Affiliation as printed
Dermatopathology Laboratory, Friedrichshafen, Germany
-
Affiliation as printed
Department of Dermatology, University Hospital Erlangen, Erlangen, Germany
-
University Hospital Regensburg
Affiliation as printed
Department of Dermatology, University Hospital Regensburg, Regensburg, Germany
-
Affiliation as printed
Department of Dermatology, University Hospital Essen, Essen, Germany
-
Affiliation as printed
Department of Dermatology, University Hospital Essen, Essen, Germany
-
Affiliation as printed
Department of Dermatology, University Hospital Würzburg, Würzburg, Germany
-
Affiliation as printed
Department of Dermatology, University Hospital Würzburg, Würzburg, Germany
-
German Cancer Research Center · National Center for Tumor Diseases
Affiliation as printed
Digital Biomarkers for Oncology Group, National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
-
German Cancer Research Center · National Center for Tumor Diseases
Affiliation as printed
Section Translational Cancer Epigenomics, Division Translational Medical Oncology, German Cancer Research Center (DKFZ) & National Center for Tumor Diseases (NCT), Heidelberg, 69120, Germany
-
German Cancer Research Center · National Center for Tumor Diseases
Affiliation as printed
Section Translational Cancer Epigenomics, Division Translational Medical Oncology, German Cancer Research Center (DKFZ) & National Center for Tumor Diseases (NCT), Heidelberg, 69120, Germany
-
Universitätsklinikum Aachen · RWTH Aachen University
Affiliation as printed
Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
-
Affiliation as printed
Private Laboratory of Dermatohistopathology, Mönchhofstraße 52, Heidelberg, 69120, Germany
-
Affiliation as printed
Anatomic Pathology Unit, Macerata General Hospital, Macerata, Italy
-
German Cancer Research Center · National Center for Tumor Diseases
Affiliation as printed
Digital Biomarkers for Oncology Group, National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
-
Titus Josef Brinker corresponding
German Cancer Research Center · National Center for Tumor Diseases
Affiliation as printed
Digital Biomarkers for Oncology Group, National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany. Electronic address: titus.brinker@dkfz.de
Digital Biomarkers for Oncology Group, National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
Cited by 108 stored of 108
No patents citing this paper on Lens.org (checked 2026-10-06).