Assessing clinical decision support system tools in precision oncology: piloting ring testing
ESMO Real World Data and Digital Oncology, vol. 13, pp. 100731
Abstract
Background: The EU4Health project PCM4EU aimed to improve survival rates and quality of life of patients with cancer based on precision cancer medicine. To achieve this, enhanced expertise and quality of molecular cancer diagnostics are key. Clinical decision support systems (CDSS) have become increasingly important after the introduction of comprehensive genomic diagnostic profiling. While external quality assessment schemes are mandatory for most diagnostic laboratory tests, similar programs for CDSS tools are currently lacking. To address this, we piloted an international ring test documenting CDSS usage, performance, and manual interpretation. Materials and methods: Twenty synthetic datasets were generated, mimicking small variant call sets from a typical targeted 500-gene panel (VCF format) across multiple cancer types (10 tumour-normal pairs and 10 tumour-only). Participants received standardised instructions via e-mail and at a virtual meeting and submitted results using a structured response form. Results: Eight laboratories from seven countries participated. All participants submitted results for the 10 tumour-only cases; one submitted results from two assessors. Tumour-only cases contained 6-18 variants where interpretation could be critical. Oncogenic calls for hotspot variants showed good agreement across the various CDSS tools applied; however, variability existed regarding reported variants and clinical interpretation. Conclusion: The pilot ring test revealed clinically relevant discrepancies between laboratories and interpreters, underscoring the need for structured external quality assessment schemes for CDSS tools in addition to the existing laboratory workflow schemes. It also highlighted several challenges related to the generation of realistic synthetic data, the design of reporting formats, the definition of ground truth, and the manual interpretation of results.
Authors 22
-
Affiliation as printed
Department of Pathology, Division of Laboratory Medicine, Oslo University Hospital, Oslo, Norway
-
Affiliation as printed
Department of Pathology, Division of Laboratory Medicine, Oslo University Hospital, Oslo, Norway
-
Karolinska University Hospital · Science for Life Laboratory · Karolinska Institutet
Affiliation as printed
Department of Oncology-Pathology, Karolinska Institutet, Science for Life Laboratory, Stockholm, Sweden
Division of Pathology, Karolinska University Hospital, Stockholm, Sweden
-
Karolinska University Hospital · Karolinska Institutet
Affiliation as printed
Clinical Genetics and Genomics, Karolinska University Hospital, Stockholm, Sweden
Department of Molecular Medicine and Surgery, Karolinska Institutet, Stockholm, Sweden
-
Heidelberg University · University Hospital Heidelberg
Affiliation as printed
Institute for Pathology, Heidelberg University Hospital, Heidelberg, Germany
-
Affiliation as printed
Hartwig Medical Foundation, Amsterdam, the Netherlands
-
Karolinska University Hospital · Science for Life Laboratory · Karolinska Institutet
Affiliation as printed
Department of Oncology-Pathology, Karolinska Institutet, Science for Life Laboratory, Stockholm, Sweden
Division of Pathology, Karolinska University Hospital, Stockholm, Sweden
-
Affiliation as printed
Center for Genomic Medicine, Rigshospitalet, Copenhagen, Denmark
-
Affiliation as printed
Department of Laboratory Genetics, Genetics and Personalized Medicine Clinic, Tartu University Hospital, Tartu, Estonia
-
Centre Léon Bérard · Centre de Recherche en Cancérologie de Lyon
Affiliation as printed
Centre de Recherche en Cancérologie de Lyon (CRCL), Centre Léon Bérard, Lyon, France
-
Centre Léon Bérard · Centre de Recherche en Cancérologie de Lyon
Affiliation as printed
Centre de Recherche en Cancérologie de Lyon (CRCL), Centre Léon Bérard, Lyon, France
-
Affiliation as printed
Department of Clinical Molecular Biology, Akershus University Hospital, Unit for Precision Medicine, Lørenskog, Norway
-
Affiliation as printed
Department of Clinical Molecular Biology, Akershus University Hospital, Unit for Precision Medicine, Lørenskog, Norway
-
Leiden University Medical Center
Affiliation as printed
Department of Medical Oncology, Leiden University Medical Centre, Leiden, the Netherlands
-
Affiliation as printed
Department of Pathology, Division of Laboratory Medicine, Oslo University Hospital, Oslo, Norway
-
Affiliation as printed
Department of Pathology, Division of Laboratory Medicine, Oslo University Hospital, Oslo, Norway
-
Universidade do Porto · IPO Porto · Instituto de Ciências Biomédicas Albel Salazar
Affiliation as printed
Department of Laboratory Genetics, Portuguese Institute of Oncology of Porto (IPO-Porto), Porto, Portugal
Porto Comprehensive Cancer Center, Porto, Portugal
School of Medicine and Biomedical Sciences (ICBAS), University of Porto, Porto, Portugal
-
Affiliation as printed
Department of Laboratory Genetics, Portuguese Institute of Oncology of Porto (IPO-Porto), Porto, Portugal
Porto Comprehensive Cancer Center, Porto, Portugal
-
Oslo University Hospital · University of Oslo
Affiliation as printed
Center for Cancer Cell Reprogramming, Institute of Clinical Medicine, University of Oslo, Oslo, Norway
Department of Tumor Biology, Institute for Cancer Research, Oslo University Hospital, Oslo, Norway
-
Affiliation as printed
Department of Tumor Biology, Institute for Cancer Research, Oslo University Hospital, Oslo, Norway
-
Lund University · Skåne University Hospital
Affiliation as printed
Department of Clinical Genetics, Pathology and Molecular Diagnostics, Skåne University Hospital, Lund, Sweden
Division of Pathology, Department of Clinical Sciences, Lund University, Lund, Sweden
-
Hege Giercksky Russnes corresponding
Oslo University Hospital · University of Oslo
Affiliation as printed
Department of Cancer Genetics, Institute for Cancer Research, Division of Cancer, Oslo University Hospital, Oslo, Norway
Department of Pathology, Division of Laboratory Medicine, Oslo University Hospital, Oslo, Norway
Institut for Clinical Medicine, University of Oslo, Oslo, Norway
Cited by 0 stored of 0
No patents citing this paper on Lens.org (checked 2026-10-11).
References 17
17 results