Developing high-quality, practical, and ethical automated L2 speaking assessments

System, vol. 134, pp. 103796

Abstract

To foster second language (L2) learners’ speaking abilities, practicing speaking regularly is necessary, including regular assessments and individual feedback to learners. However, in current classroom settings, practicing and assessing speaking are often neglected. For teachers, it is particularly hard to provide individualized feedback on speaking, speaking being a loud and transient phenomenon. Additionally, recording speech and thus providing individual assessments and feedback based on recordings is highly time-consuming. Therefore, to alleviate the amount of work needed for individual assessments, teachers and learners would be helped with automated speaking assessments. In this paper, we first describe the requirements for high-quality, practical, and ethical tools for automated scoring of and feedback on L2 speaking performances. Subsequently, we describe and evaluate existing tools of automated L2 speaking assessment. We conclude that none of the described tools meet all the identified requirements. Combining insights from the AI-based assessment framework (Fang et al., 2023) with an educational design approach, we offer recommendations intended to guide computational linguists together with researchers and practitioners in education and assessment on how to successfully integrate computational research with educational design. The goal of such future research is to develop generative AI (GenAI)-based systems that are technically sound, ethically responsible, and likely to be adopted in educational practice.

Authors 3

1. Nivja H. de Jong corresponding Aachen

   Leiden University

   Affiliation as printed

   Leiden University Centre for Linguistics, Leiden University, the Netherlands

   Leiden University Graduate School of Teaching, Leiden University, the Netherlands

2. Stephan Raaijmakers Aachen

   Leiden University

   Affiliation as printed

   Leiden University Centre for Linguistics, Leiden University, the Netherlands

3. Dineke E.H. Tigelaar Aachen

   Leiden University

   Affiliation as printed

   Leiden University Graduate School of Teaching, Leiden University, the Netherlands