Agentic retrieval-augmented reasoning reshapes collective reliability under model variability in radiology question answering
Abstract
Agentic retrieval-augmented reasoning pipelines are increasingly used to structure how large language models (LLMs) incorporate external evidence in clinical decision support. These systems iteratively retrieve curated domain knowledge and synthesize it into structured reports before answer selection. Although such pipelines can improve performance, their impact on reliability under model variability remains unclear. In real-world deployment, heterogeneous models may align, diverge, or synchronize errors in ways not captured by accuracy. We evaluated 34 LLMs on 169 expert-curated publicly available radiology questions, comparing zero-shot inference with a radiology-specific multi-step agentic retrieval condition in which all models received identical structured evidence reports derived from curated radiology knowledge. Agentic inference reduced inter-model decision dispersion (median entropy 0.48 vs. 0.13) and increased robustness of correctness across models (mean 0.74 vs. 0.81). Majority consensus also increased overall (P<0.001). Consensus strength and robust correctness remained correlated under both strategies (\r{ho}=0.88 for zero-shot; \r{ho}=0.87 for agentic), although high agreement did not guarantee correctness. Response verbosity showed no meaningful association with correctness. Among 572 incorrect outputs, 72% were associated with moderate or high clinically assessed severity, although inter-rater agreement was low (\k{appa}=0.02). Agentic retrieval therefore was associated with more concentrated decision distributions, stronger consensus, and higher cross-model robustness of correctness. These findings suggest that evaluating agentic systems through accuracy or agreement alone may not always be sufficient, and that complementary analyses of stability, cross-model robustness, and potential clinical impact are needed to characterize reliability under model variability.
Authors 12
-
Affiliation as printed
( ) Lab for AI in Medicine , RWTH Aachen University , Aachen , Germany. (
-
Friedrich-Alexander-Universität Erlangen-Nürnberg
Affiliation as printed
) Pattern Recognition Lab , Friedrich-Alexander-Universität Erlangen-Nürnberg , Erlangen , Germany
-
Friedrich-Alexander-Universität Erlangen-Nürnberg
Affiliation as printed
) Pattern Recognition Lab , Friedrich-Alexander-Universität Erlangen-Nürnberg , Erlangen , Germany
-
TUM Klinikum · Technical University of Munich
Affiliation as printed
) Department of Diagnostic and Interventional Radiology , TUM University Klinikum , School of Medicine and Health , Klinikum rechts der Isar , Technical University of Munich , Munich , Germany
-
Felix Barajas Ordonez Aachen Lab for AI in Medicine Department of Diagnostic and Interventional Radiology
Universitätsklinikum Aachen · RWTH Aachen University
Affiliation as printed
( ) Lab for AI in Medicine , RWTH Aachen University , Aachen , Germany. (
) Department of Diagnostic and Interventional Radiology , University Hospital RWTH Aachen , Aachen , Germany. (
-
Friedrich-Alexander-Universität Erlangen-Nürnberg · Universitätsklinikum Erlangen
Affiliation as printed
) Institute of Radiology , University Hospital Erlangen , Erlangen , Germany. (
) Pattern Recognition Lab , Friedrich-Alexander-Universität Erlangen-Nürnberg , Erlangen , Germany
-
Mahshad Lotfinia Aachen Lab for AI in Medicine Department of Diagnostic and Interventional Radiology
Universitätsklinikum Aachen · RWTH Aachen University
Affiliation as printed
( ) Lab for AI in Medicine , RWTH Aachen University , Aachen , Germany. (
) Department of Diagnostic and Interventional Radiology , University Hospital RWTH Aachen , Aachen , Germany. (
-
Friedrich-Alexander-Universität Erlangen-Nürnberg
Affiliation as printed
) Erlangen National High Performance Computing Center , Friedrich-Alexander-Universität Erlangen-Nürnberg , Erlangen , Germany
) Pattern Recognition Lab , Friedrich-Alexander-Universität Erlangen-Nürnberg , Erlangen , Germany
-
TUM Klinikum · Deutsches Herzzentrum München · Technical University of Munich
Affiliation as printed
) Department of Cardiovascular Radiology and Nuclear Medicine , TUM University Clinic , School of Medicine and Health , German Heart Center , Technical University of Munich , Munich , Germany
) Department of Diagnostic and Interventional Radiology , TUM University Klinikum , School of Medicine and Health , Klinikum rechts der Isar , Technical University of Munich , Munich , Germany
-
Universitätsklinikum Aachen · RWTH Aachen University
Affiliation as printed
( ) Lab for AI in Medicine , RWTH Aachen University , Aachen , Germany. (
) Department of Diagnostic and Interventional Radiology , University Hospital RWTH Aachen , Aachen , Germany. (
Cited by 0 stored of 0
No patents citing this paper on Lens.org (checked 2026-10-06).