Computer Adaptive vs. Non-adaptive Medical Progress Testing: Feasibility, Test Performance, and Student Experiences

Perspectives on Medical Education, vol. 13, pp. 406–416

Abstract

Background: Computerized adaptive testing tailors test items to students' abilities by adapting difficulty level. This more efficient, and reliable assessment form may provide advantages over a conventional medical progress test (PT). Prior to our study, a direct comparison of students' performance on a computer adaptive progress test (CA-PT) and a conventional PT, which is crucial for nationwide implementation of the CA-PT, was missing. Therefore, we assessed the correlation between CA-PT and conventional PT test performance and explored the feasibility and student experiences of CA-PT in a large medical cohort. Methods: In this cross-over study medical students (n = 1432) of three Dutch medical schools participated in both a conventional PT and CA-PT. They were stratified to start with either a conventional PT or CA-PT to determine test performance. Student motivation, engagement and experiences were assessed by questionnaires in students from seven Dutch medical schools. Parallel-forms reliability was assessed using the Pearson correlation coefficient. Results: A strong correlation was found (0.834) between conventional PT and CA-PT test performance. The CA-PT was administered without system performance issues and was completed in a median time of 83 minutes (67-102 minutes). Questionnaire response rate was 31.7% (526/1658). Despite a higher experienced difficulty, most students reported persistence, adequate task management and good focus during the CA-PT. Conclusions: CA-PT provides a reliable estimation of students' ability level in less time than a conventional non-adaptive PT and is feasible in students throughout the entire medical curriculum. Despite the strong correlation between PT scores, students found the CA-PT more challenging.

Authors 10

1. Elise V. van Wijk Aachen Center for Innovation in Medical Education Department of Gastroenterology and Hepatology

   Leiden University Medical Center

   Affiliation as printed

   Center for Innovation in Medical Education, Leiden University Medical Center, the Netherlands

   Center for in Medical Education , University Medical , the

   Department of Gastroenterology and Hepatology, Leiden University Medical Center, Albinusdreef 2, 2333 ZA, Leiden, The Netherlands

2. Jeroen Donkers Aachen Department of Gastroenterology and Hepatology

   Leiden University Medical Center · Maastricht University

   Affiliation as printed

   School of Health Professions Education, Faculty of Health, Medicine and Life Sciences, Maastricht University, the Netherlands

   Center for in Medical Education , University Medical , the

   Department of Gastroenterology and Hepatology, Leiden University Medical Center, Albinusdreef 2, 2333 ZA, Leiden, The Netherlands

3. Peter C. J. de Laat Aachen Department of Gastroenterology and Hepatology

   Leiden University Medical Center · Erasmus MC

   Affiliation as printed

   Department of Pediatrics, Erasmus Medical Center, Rotterdam, the Netherlands

   Center for in Medical Education , University Medical , the

   Department of Gastroenterology and Hepatology, Leiden University Medical Center, Albinusdreef 2, 2333 ZA, Leiden, The Netherlands

4. Ariadne A. Meiboom Aachen Department of Gastroenterology and Hepatology

   Leiden University Medical Center · Amsterdam University Medical Centers

   Affiliation as printed

   Department of General Practice and Elderly Care Medicine, Amsterdam University Medical Center, Amsterdam, the Netherlands

   Center for in Medical Education , University Medical , the

   Department of Gastroenterology and Hepatology, Leiden University Medical Center, Albinusdreef 2, 2333 ZA, Leiden, The Netherlands

5. Bram Jacobs Aachen Department of Gastroenterology and Hepatology

   University Medical Center Groningen · University of Groningen · Leiden University Medical Center

   Affiliation as printed

   Department of Neurology, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands

   Center for in Medical Education , University Medical , the

   Department of Gastroenterology and Hepatology, Leiden University Medical Center, Albinusdreef 2, 2333 ZA, Leiden, The Netherlands

6. Jan Hindrik Ravesloot Aachen Department of Gastroenterology and Hepatology

   Leiden University Medical Center · Amsterdam University Medical Centers

   Affiliation as printed

   Department of Physiology, Amsterdam University Medical Center, Amsterdam, the Netherlands

   Center for in Medical Education , University Medical , the

   Department of Gastroenterology and Hepatology, Leiden University Medical Center, Albinusdreef 2, 2333 ZA, Leiden, The Netherlands

7. René A. Tio Aachen Department of Gastroenterology and Hepatology

   Leiden University Medical Center · Catharina Ziekenhuis

   Affiliation as printed

   Department of Cardiology, Catharina Hospital Eindhoven, Eindhoven, the Netherlands

   Center for in Medical Education , University Medical , the

   Department of Gastroenterology and Hepatology, Leiden University Medical Center, Albinusdreef 2, 2333 ZA, Leiden, The Netherlands

8. Cees van der Vleuten Aachen Department of Gastroenterology and Hepatology

   Leiden University Medical Center · Maastricht University

   Affiliation as printed

   Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands

   Center for in Medical Education , University Medical , the

   Department of Gastroenterology and Hepatology, Leiden University Medical Center, Albinusdreef 2, 2333 ZA, Leiden, The Netherlands

9. Alexandra M. J. Langers corresponding Aachen Department of Gastroenterology and Hepatology

   Leiden University Medical Center

   Affiliation as printed

   Department of Gastroenterology and Hepatology, Leiden University Medical Center, Leiden, the Netherlands

   Center for in Medical Education , University Medical , the

   Department of Gastroenterology and Hepatology, Leiden University Medical Center, Albinusdreef 2, 2333 ZA, Leiden, The Netherlands

10. André J. A. Bremers Aachen Department of Gastroenterology and Hepatology

    Leiden University Medical Center · Radboud University Medical Center

    Affiliation as printed

    Department of Surgery, Radboud University Medical Center, Nijmegen, the Netherlands

    Center for in Medical Education , University Medical , the

    Department of Gastroenterology and Hepatology, Leiden University Medical Center, Albinusdreef 2, 2333 ZA, Leiden, The Netherlands