Context Matters in LLM-Assisted Qualitative Data Analysis: Workflow Development and Multidimensional Evaluation in Health Research
Abstract
Abstract Large language models (LLMs) are increasingly used for qualitative analysis, but strong language performance does not guarantee strong interpretation when meaning depends on method and context. We developed a context-specific workflow that translated framework analysis into bounded, sequential LLM-supported tasks and applied it to 20 Chinese-language interviews on fear of dementia (approximately 150,000 characters). Four multilingual LLM-generated outputs and a previous researcher-only analysis were blindly evaluated by 16 relevant experts across different analytical domains and qualitative quality dimensions (496 expert-item records). All LLMs completed the workflow, but rankings varied markedly. Qwen-output showed the most consistently favourable overall profile, while Claude-output was most often ranked first in individual domain–dimension evaluations. The researcher analysis received both highly favourable and highly unfavourable rankings; no output was consistently preferred. LLM-assisted qualitative analysis should be evaluated within its research contexts, with transparent workflows, multidimensional assessment, and continued researcher oversight.
Authors 9
-
Huohuo Dai corresponding Aachen Department of Public Health and Primary Care National eHealth Living Lab
Leiden University Medical Center
Affiliation as printed
Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, the Netherlands
National eHealth Living Lab, Leiden University Medical Center, Leiden, the Netherlands
-
University Medical Center Groningen · University of Groningen · Chinese Academy of Medical Sciences & Peking Union Medical College
Affiliation as printed
Department of Epidemiology, University Medical Center Groningen, University of Groningen, Groningen, the Netherlands
School of Health Policy and Management, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, Beijing, China
-
María José Villalobos-Quesada Aachen Department of Public Health and Primary Care National eHealth Living Lab
Leiden University Medical Center
Affiliation as printed
Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, the Netherlands
National eHealth Living Lab, Leiden University Medical Center, Leiden, the Netherlands
-
Yee Man Ng Aachen Department of Public Health and Primary Care Leiden Institute of Advanced Computer Science
Leiden University · Leiden University Medical Center
Affiliation as printed
Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, the Netherlands
Leiden Institute of Advanced Computer Science, Leiden University, Leiden, the Netherlands
-
Jiska Joëlle Aardoom Aachen Department of Public Health and Primary Care National eHealth Living Lab
Leiden University Medical Center
Affiliation as printed
Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, the Netherlands
National eHealth Living Lab, Leiden University Medical Center, Leiden, the Netherlands
-
Leiden University Medical Center
Affiliation as printed
Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, the Netherlands
National eHealth Living Lab, Leiden University Medical Center, Leiden, the Netherlands
-
Affiliation as printed
School of Nursing, Guangzhou Medical University, Guangzhou, China
-
Marco René Spruit Aachen Department of Public Health and Primary Care Leiden Institute of Advanced Computer Science
Leiden University · Leiden University Medical Center
Affiliation as printed
Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, the Netherlands
Leiden Institute of Advanced Computer Science, Leiden University, Leiden, the Netherlands
-
Niels Henrik Chavannes Aachen Department of Public Health and Primary Care National eHealth Living Lab
Leiden University Medical Center
Affiliation as printed
Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, the Netherlands
National eHealth Living Lab, Leiden University Medical Center, Leiden, the Netherlands
Cited by 0 stored of 1
No patents citing this paper on Lens.org (checked 2026-10-11).