Learning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School Tutoring

arXiv (Cornell University)

Abstract

LLMs can personalize education, although current static-prompt tutoring systems struggle to adapt to diverse academic disciplines. We develop and test a system with subject-aware prompting, based on 14 pedagogical features (e.g., tutor scaffolding, student understanding) extracted from raw transcripts. We first train a prompt routing model in a simulation environment, and then deploy it for online adaptation with actual high-school students. The simulation benchmark shows the router outperforming two static baselines ($0.694$ vs. $0.647$ and $0.64$, $p<0.001$). A/B testing ($N=656$ conversations from 359 students) shows sim-to-real transfer where the model switches from analytical to scaffolding learning strategies. Our adaptive prompt selection mechanism improves instructional efficiency, maintains pedagogical quality and reduces interactions by around 3 turns ($p=0.007$). While a greedy router achieves a comparable exercise conversion rate with the baseline ($19.1\%$ vs. $19.6\%$), a stochastic router that samples strategies leads to a higher conversion rate ($28.1\%$).

Authors 4

1. Po-Chin Chang Aachen

   Leiden University

   Affiliation as printed

   Leiden University

2. Nicholas Hogan
3. Aske Plaat Aachen

   Leiden University

   Affiliation as printed

   Leiden University

4. Michiel van der Meer Aachen

   Leiden University

   Affiliation as printed

   Leiden University