Multi-step retrieval and reasoning improves radiology question answering with large language models

npj Digital Medicine, vol. 8, pp. 790

Abstract

Abstract Large language models (LLMs) show promise for radiology decision support, yet conventional retrieval-augmented generation (RAG) relies on single-step retrieval and struggles with complex reasoning. We introduce radiology Retrieval and Reasoning (RaR), a multi-step retrieval framework that iteratively summarizes clinical questions, retrieves evidence, and synthesizes answers. We evaluated 25 LLMs spanning general-purpose, reasoning-optimized, and clinically fine-tuned models (0.5B → 670B parameters) on 104 expert-curated radiology questions and an independent set of 65 real radiology board-exam questions. RaR significantly improved mean diagnostic accuracy versus zero-shot prompting (75% vs. 67%; P = 1.1 × 10 −7 ) and conventional online RAG (75% vs. 69%; P = 1.9 × 10 −6 ). Gains were largest in mid-sized and small models (e.g., Mistral Large: 72% → 81%), while very large models showed minimal change. RaR reduced hallucinations and provided clinically relevant evidence in 46% of cases, improving factual grounding. These results show that multi-step retrieval enhances diagnostic reliability, especially in deployable mid-sized LLMs. Code, datasets, and RaR are publicly available.

Authors 12

1. Sebastian Wind

   Friedrich-Alexander-Universität Erlangen-Nürnberg

   Affiliation as printed

   Erlangen National High Performance Computing Center, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

   Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

2. Jeta Sopa

   Friedrich-Alexander-Universität Erlangen-Nürnberg

   Affiliation as printed

   Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

3. Daniel Truhn Aachen Department of Diagnostic and Interventional Radiology

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany

4. Mahshad Lotfinia Aachen Department of Diagnostic and Interventional Radiology

   Universitätsklinikum Aachen · RWTH Aachen University

   Affiliation as printed

   Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany

5. Tri-Thien Nguyen

   Friedrich-Alexander-Universität Erlangen-Nürnberg · Universitätsklinikum Erlangen

   Affiliation as printed

   Institute of Radiology, University Hospital Erlangen, Erlangen, Germany

   Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

6. Keno K. Bressem

   TUM Klinikum · Deutsches Herzzentrum München · Technical University of Munich

   Affiliation as printed

   Department of Cardiovascular Radiology and Nuclear Medicine, TUM University Clinic, School of medicine and Health, German Heart Center, Technical University of Munich, Munich, Germany

   Department of Diagnostic and Interventional Radiology, TUM University Clinic, School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Munich, Germany

7. Lisa C. Adams

   TUM Klinikum · Technical University of Munich

   Affiliation as printed

   Department of Diagnostic and Interventional Radiology, TUM University Clinic, School of Medicine and Health, Klinikum rechts der Isar, Technical University of Munich, Munich, Germany

8. Mirabela Rusu

   Stanford Medicine · Stanford University

   Affiliation as printed

   Department of Radiology, Stanford University, Stanford, CA, USA

   Department of Urology, Stanford University, Stanford, CA, USA

9. Harald Koestler

   Friedrich-Alexander-Universität Erlangen-Nürnberg

   Affiliation as printed

   Chair of Computer Science 10, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

   Erlangen National High Performance Computing Center, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

10. Gerhard Wellein

    Friedrich-Alexander-Universität Erlangen-Nürnberg

    Affiliation as printed

    Erlangen National High Performance Computing Center, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

11. Andreas Maier

    Friedrich-Alexander-Universität Erlangen-Nürnberg

    Affiliation as printed

    Erlangen National High Performance Computing Center, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

    Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany

12. Soroosh Tayebi Arasteh corresponding Aachen Department of Diagnostic and Interventional Radiology

    Friedrich-Alexander-Universität Erlangen-Nürnberg · Universitätsklinikum Aachen · Stanford Medicine · RWTH Aachen University · Stanford University

    Affiliation as printed

    Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany. soroosh.arasteh@rwth-aachen.de

    Department of Radiology, Stanford University, Stanford, CA, USA. soroosh.arasteh@rwth-aachen.de

    Department of Urology, Stanford University, Stanford, CA, USA. soroosh.arasteh@rwth-aachen.de

    Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany. soroosh.arasteh@rwth-aachen.de