Evaluating auditory selective attention in classroom scenarios using audiovisual virtual reality : from auditory to audiovisual scenarios

RWTH Publications (RWTH Aachen)

Abstract

Auditory selective attention enables humans to focus on relevant sound sources while suppressing distractions, a skill crucial in everyday environments. For example, in a classroom, a student may need to attend to the teacher’s instructions while filtering out whispering peers, moving chairs, pen noises, and even traffic sounds from outside. Despite decades of research, little is known about how complex audio-visual factors, realistic noise, and room acoustics influence this ability. Recent advances in consumer-grade virtual reality (VR) offer an opportunity to study auditory cognition in closer-to-real-life settings.This thesis investigates the voluntary switching of auditory selective attention in a VR classroom. An established child-appropriate audio-only paradigm, in which participants classify animal names presented from different spatial positions, was transferred into VR using a visually realistic classroom environment. A proof-of-concept study confirmed the VR paradigm’s feasibility, with participants generally responding faster and more accurately, likely reflecting increased engagement or motivation in VR.Cross-modal integration was examined in a follow-up study, testing for audio-visual congruence effects. No consistent effects were observed, highlighting the challenges of directly translating traditional paradigms into immersive VR and the need to design VR-native tasks that leverage interaction, spatial exploration, and multisensory engagement.The acoustic environment was systematically enhanced in subsequent studies. A complex classroom noise scene, with and without intelligible speech, impaired task performance and increased listening effort, demonstrating the importance of realistic noise for auditory cognition research. Room acoustics were also manipulated by extending stimuli from single words to sentences and simulating favorable ($T_{30} = 0.48 \ \mathrm{s}$) and unfavorable ($T_{30} = 1.52 \ \mathrm{s}$) reverberation times. While overall performance and spatial separation were affected by reverberation, longer stimuli facilitated the reorientation of attention, reducing the expected attention switching costs.Together, these studies show that investigating auditory selective attention in closer-to-real-life VR environments requires more than simply adapting existing laboratory paradigms. Complex acoustic properties, visual context, and cognitive demands interact in ways not captured by traditional setups, underlining the need for VR-specific paradigms to study real-world behavior. The findings emphasize the importance of interdisciplinary approaches and the integration of audio-visual, acoustic, and task-related factors in the design of immersive experiments.

Authors 1

1. Carolin Breuer corresponding Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen