Artificial Intelligence in Plastic Surgery Education: Insights from Parallel Turkish and English Versions of a Board Examination

Applied Sciences, vol. 16, pp. 7559

Abstract

Background: Large language models (LLMs) increasingly pass medical board examinations and aid clinical knowledge retrieval and decision support; validating their specialist knowledge is a prerequisite for safe use. Two limitations weaken existing evidence: many studies reuse public questions that may be in training data, and most evaluations use only English-language examinations. Plastic surgery serves here as a representative, highly specialized subfield of medicine. Methods: We used the non-public board examination of the Turkish Society of Plastic, Reconstructive, and Aesthetic Surgeons (TSPRAS): 100 single-best-answer (SBA) items in official parallel Turkish and English versions. Six contemporary LLMs from three developers, in matched free and paid tiers, completed six runs per language (7200 responses) under consistent English instruction. Accuracy, inter-run reliability, and cross-language and tier differences were analyzed. Results: All models exceeded the 60% passing threshold (71.5–86.4% aggregate), with almost perfect inter-run agreement (Cohen’s κ ≥ 0.843). Question language produced no significant difference after correction, and item difficulty correlated strongly across languages (Pearson r = 0.928); divergent items reflected content, not language. Paid variants offered only incremental advantages. Conclusions: Current LLMs showed no significant accuracy difference between the Turkish and English examination versions under a constant English prompt, supporting supervised study-aid use on this MCQ format.

Authors 6

1. Ibrahim Güler corresponding

   Friedrich-Alexander-Universität Erlangen-Nürnberg · Otto-von-Guericke-Universität Magdeburg

   Affiliation as printed

   Department of Health Management, Friedrich-Alexander-Universität Erlangen-Nürnberg, 90403 Nürnberg, Germany

   Department of Plastic, Aesthetic and Hand Surgery, Otto-von-Guericke University, 39120 Magdeburg, Germany

2. Armin Kraus

   Otto-von-Guericke-Universität Magdeburg

   Affiliation as printed

   Department of Plastic, Aesthetic and Hand Surgery, Otto-von-Guericke University, 39120 Magdeburg, Germany

3. Gerrit Grieb Aachen Department of Plastic Surgery and Hand Surgery Medical Faculty

   Gemeinschaftskrankenhaus Havelhöhe · RWTH Aachen University

   Affiliation as printed

   Department of Plastic Surgery and Hand Surgery, Gemeinschaftskrankenhaus Havelhoehe, 14089 Berlin, Germany

   Department of Plastic Surgery and Hand Surgery, Medical Faculty, Rheinisch-Westfälische Technische Hochschule Aachen University, 52074 Aachen, Germany

4. Uzay Cambaz

   University of Tübingen

   Affiliation as printed

   Faculty of Medicine, Eberhard Karls University of Tübingen, 72076 Tübingen, Germany

5. Henrik Stelling

   Friedrich-Alexander-Universität Erlangen-Nürnberg

   Affiliation as printed

   Department of Health Management, Friedrich-Alexander-Universität Erlangen-Nürnberg, 90403 Nürnberg, Germany

   Practices for Nuclear Medicine, 12157 Berlin, Germany

6. Cenk Demirdöver
   Affiliation as printed

   European Board of Plastic, Reconstructive and Aesthetic Surgery (EBOPRAS), 9700 RB Groningen, The Netherlands

   European Society of Plastic, Reconstructive and Aesthetic Surgery (ESPRAS), 5001 Aarau, Switzerland

   Turkish Society of Plastic, Reconstructive and Aesthetic Surgeons (TSPRAS), 06530 Ankara, Turkey