The Effect of the Question Mark Option in Progress Testing: A Large-Scale Longitudinal Study
Perspectives on Medical Education, vol. 14, pp. 891–904
Abstract
Introduction: Formula scoring, widely used in medical progress tests (PT), includes a question mark option to discourage guessing, but this feature may disadvantage risk-averse students and bias results due to test-taking strategies. To enhance reliability and more accurately assess ability, Dutch medical schools recently transitioned to a computer adaptive-PT (CA-PT) based on Item Response Theory, which adjusts question difficulty dynamically, excluding the question mark option. This provided a unique opportunity to evaluate the impact of the question mark option in a large cohort. We specifically explored the relationship between question mark use in conventional PT and performance on CA-PT. Methods: Retrospective data from medical students across seven faculties who took both PT formats were analyzed. Z-scores for total score and question mark score (number of unanswered questions) in the conventional PT, and theta score for the CA-PT were assessed. A linear model assessed the effect of the question mark score on theta, corrected for the conventional PT-score. Cluster analysis explored student subgroups per year. Results: Students with similar conventional PT scores who left more questions unanswered on the conventional PT generally performed better on CA-PT. This effect diminished as students advanced through their studies. Cluster analysis revealed a variable effect between different students, most pronounced in year 4, and a reverse effect in year 5. Discussion: Question mark option use significantly impacted student performance on PT, with a remarkable variability among students. This variability suggests that formula scoring captures more than knowledge alone, highlighting the need to align scoring methods with intended assessment goals.
Authors 11
-
Leiden University Medical Center
Affiliation as printed
Center for Innovation in Medical Education, Leiden University Medical Center, The Netherlands
-
Affiliation as printed
School of Health Professions Education, Faculty of Health, Medicine and Life Sciences, Maastricht University, The Netherlands
-
Affiliation as printed
Department of Pediatrics, Erasmus Medical Center, Rotterdam, The Netherlands
-
Amsterdam University Medical Centers
Affiliation as printed
Department of General Practice and Elderly Care Medicine, Amsterdam University Medical Center, Amsterdam, The Netherlands
-
University Medical Center Groningen · University of Groningen
Affiliation as printed
Department of Neurology, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands
-
Amsterdam University Medical Centers
Affiliation as printed
Department of Physiology, Amsterdam University Medical Center, Amsterdam, The Netherlands
-
Affiliation as printed
Department of Cardiology, Catharina Hospital Eindhoven, Eindhoven, The Netherlands
-
University Medical Center Utrecht
Affiliation as printed
Education Centre, Department of Medical Education, University Medical Center Utrecht, Utrecht, The Netherlands
-
Affiliation as printed
Department of Internal Medicine, Division of Nephrology, Maastricht University, The Netherlands
-
Radboud University Medical Center
Affiliation as printed
Department of Surgery, Radboud University Medical Center, Nijmegen, The Netherlands
-
Leiden University Medical Center
Affiliation as printed
Department of Gastroenterology and Hepatology, Leiden University Medical Center, Leiden, The Netherlands
Cited by 0 stored of 0
No patents citing this paper on Lens.org (checked 2026-10-11).