Prompt injection attacks on vision language models in oncology
Nature Communications, vol. 16, pp. 1239
Abstract
Vision-language artificial intelligence models (VLMs) possess medical knowledge and can be employed in healthcare in numerous ways, including as image interpreters, virtual scribes, and general decision support systems. However, here, we demonstrate that current VLMs applied to medical tasks exhibit a fundamental security flaw: they can be compromised by prompt injection attacks. These can be used to output harmful information just by interacting with the VLM, without any access to its parameters. We perform a quantitative study to evaluate the vulnerabilities to these attacks in four state of the art VLMs: Claude-3 Opus, Claude-3.5 Sonnet, Reka Core, and GPT-4o. Using a set of N = 594 attacks, we show that all of these models are susceptible. Specifically, we show that embedding sub-visual prompts in manifold medical imaging data can cause the model to provide harmful output, and that these prompts are non-obvious to human observers. Thus, our study demonstrates a key vulnerability in medical VLMs which should be mitigated before widespread clinical adoption.
Authors 8
-
Universitätsklinikum Aachen · Else Kröner Fresenius Center for Digital Health · RWTH Aachen University
Affiliation as printed
Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
-
Heidelberg University · University Hospital Heidelberg · National Center for Tumor Diseases · Else Kröner Fresenius Center for Digital Health
Affiliation as printed
Department of Medical Oncology, National Center for Tumor Diseases (NCT), Heidelberg University Hospital, Heidelberg, Germany
Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
-
Heidelberg University · Medizinische Fakultät Mannheim · Else Kröner Fresenius Center for Digital Health
Affiliation as printed
Department of Medicine II, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
-
Universitätsklinikum Aachen · Else Kröner Fresenius Center for Digital Health · RWTH Aachen University
Affiliation as printed
Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
-
Affiliation as printed
Digital Biomarkers for Oncology Group, German Cancer Research Center, Heidelberg, Germany
-
University Medical Center of the Johannes Gutenberg University Mainz
Affiliation as printed
Institute of Pathology, University Medical Center Mainz, Mainz, Germany
-
Affiliation as printed
Department of Diagnostic and Interventional Radiology, University Hospital Aachen, Aachen, Germany
-
Jakob Nikolas Kather corresponding
Heidelberg University · University Hospital Heidelberg · National Center for Tumor Diseases · University Hospital Carl Gustav Carus · Else Kröner Fresenius Center for Digital Health · Technische Universität Dresden
Affiliation as printed
Department of Medical Oncology, National Center for Tumor Diseases (NCT), Heidelberg University Hospital, Heidelberg, Germany. Jakob.Kather@ukdd.de
Department of Medicine I, University Hospital Dresden, Dresden, Germany. Jakob.Kather@ukdd.de
Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany. Jakob.Kather@ukdd.de
Department of Medical Oncology, National Center for Tumor Diseases (NCT), Heidelberg University Hospital, Heidelberg, Germany
Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
Cited by 56 stored of 58
No patents citing this paper on Lens.org (checked 2026-10-06).