Exploring cross-modal perception in a virtual classroom: the effect of visual stimuli on auditory selective attention

Frontiers in Psychology, vol. 16, pp. 1512851

Abstract

In virtual reality research, distinguishing between auditory and visual influences on perception has become increasingly challenging. To study auditory selective attention in more close-to-real-life settings, an auditory task was adapted to a virtual classroom. The new environment suggested evidence of increased attention, possibly introduced by the visual representation, gamification effects, and immersion. This could engage participants more effectively. To delve deeper into the impact of cross-modal effects, the paradigm was extended by visual stimuli. Participants were initially tasked with directing their auditory attention to a cued spatial position and categorizing animal names played from that position while ignoring distracting sounds. Animal pictures introduced in Experiment 1 were either congruent or incongruent with the auditory target stimuli, thus either supporting or competing with the auditory information. The concurrent presentation of animal pictures with the animal names increased response times compared to the auditory condition, and incongruent visual stimuli increased response times more than congruent ones. Fewer errors were made with congruent compared to incongruent pictures, and error rates of the auditory condition fell in between. When the visual stimulus was presented 750 ms or 500 ms before the auditory stimuli in Experiment 2, auditory and visual congruence effects interacted. In the 500 ms case, visually congruent stimuli decreased error rates in auditory incongruent trials. Conversely, visually incongruent stimuli decreased error rates on auditory incongruent trials at 750 ms. This reversal of effects suggests a positive priming effect at 500 ms and a semantic inhibition of return effect at 750 ms. Taken together, these findings indicate that cross-modal priming is at least partially different from multisensory integration.

Authors 6

1. Carolin Breuer corresponding Aachen Institute for Hearing Technology and Acoustics

   RWTH Aachen University

   Affiliation as printed

   Institute for Hearing Technology and Acoustics, RWTH Aachen University, Aachen, Germany

2. Larissa Leist

   Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau

   Affiliation as printed

   Biological Psychology, RPTU Kaiserslautern-Landau, Kaiserslautern, Germany

   Cognitive and Developmental Psychology, RPTU Kaiserslautern-Landau, Kaiserslautern, Germany

3. Stephan Fremerey

   Technische Universität Ilmenau

   Affiliation as printed

   Audiovisual Technology Group, Technische Universität Ilmenau, Ilmenau, Germany

4. Alexander Raake Aachen Institute for Communications Engineering

   Technische Universität Ilmenau · RWTH Aachen University

   Affiliation as printed

   Audiovisual Technology Group, Technische Universität Ilmenau, Ilmenau, Germany

   Institute for Communications Engineering, RWTH Aachen University, Aachen, Germany

5. Maria Klatte

   Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau

   Affiliation as printed

   Cognitive and Developmental Psychology, RPTU Kaiserslautern-Landau, Kaiserslautern, Germany

6. Janina Fels Aachen Institute for Hearing Technology and Acoustics

   RWTH Aachen University

   Affiliation as printed

   Institute for Hearing Technology and Acoustics, RWTH Aachen University, Aachen, Germany