Auditory-visual scenes for hearing research

Acta Acustica, vol. 6, pp. 55

Abstract

While experimentation with synthetic stimuli in abstracted listening situations has a long standing and successful history in hearing research, an increased interest exists on closing the remaining gap towards real-life listening by replicating situations with high ecological validity in the lab. This is important for understanding the underlying auditory mechanisms and their relevance in real-life situations as well as for developing and evaluating increasingly sophisticated algorithms for hearing assistance. A range of ‘classical’ stimuli and paradigms have evolved to de-facto standards in psychoacoustics, which are simplistic and can be easily reproduced across laboratories. While they ideally allow for across laboratory comparisons and reproducible research, they, however, lack the acoustic stimulus complexity and the availability of visual information as observed in everyday life communication and listening situations. This contribution aims to provide and establish an extendable set of complex auditory-visual scenes for hearing research that allow for ecologically valid testing in realistic scenes while also supporting reproducibility and comparability of scientific results. Three virtual environments are provided (underground station, pub, living room), consisting of a detailed visual model, an acoustic geometry model with acoustic surface properties as well as a set of acoustic measurements in the respective real-world environments. The current data set enables i) audio–visual research in a reproducible set of environments, ii) comparison of room acoustic simulation methods with “ground truth” acoustic measurements, iii) a condensation point for future extensions and contributions for developments towards standardized test cases for ecologically valid hearing research in complex scenes.

Authors 10

1. Steven van de Par

   Carl von Ossietzky Universität Oldenburg · Hearing4all

   Affiliation as printed

   Carl-von-Ossietzky Universität, Oldenburg, Dept. Medical Physics and Acoustics, Cluster of Excellence “Hearing4all”, Carl-von-Ossietzky-Str. 9–11, 26129 Oldenburg, Germany

   Carl-von-Ossietzky Universität, Oldenburg, Dept. Medical Physics and Acoustics, Cluster of Excellence "Hearing4all", Carl-von-Ossietzky-Str. 9–11, 26129 Oldenburg, Germany

2. Stephan D. Ewert

   Carl von Ossietzky Universität Oldenburg · Hearing4all

   Affiliation as printed

   Carl-von-Ossietzky Universität, Oldenburg, Dept. Medical Physics and Acoustics, Cluster of Excellence “Hearing4all”, Carl-von-Ossietzky-Str. 9–11, 26129 Oldenburg, Germany

   Carl-von-Ossietzky Universität, Oldenburg, Dept. Medical Physics and Acoustics, Cluster of Excellence "Hearing4all", Carl-von-Ossietzky-Str. 9–11, 26129 Oldenburg, Germany

3. Ľuboš Hládek

   Technical University of Munich

   Affiliation as printed

   Technical University of Munich, Audio Information Processing, Department of Electrical and Computer Engineering, Theresienstr. 90, 80333 München, Germany

4. Christoph Kirsch

   Carl von Ossietzky Universität Oldenburg · Hearing4all

   Affiliation as printed

   Carl-von-Ossietzky Universität, Oldenburg, Dept. Medical Physics and Acoustics, Cluster of Excellence “Hearing4all”, Carl-von-Ossietzky-Str. 9–11, 26129 Oldenburg, Germany

   Carl-von-Ossietzky Universität, Oldenburg, Dept. Medical Physics and Acoustics, Cluster of Excellence "Hearing4all", Carl-von-Ossietzky-Str. 9–11, 26129 Oldenburg, Germany

5. Julia Schütze

   Carl von Ossietzky Universität Oldenburg · Hearing4all

   Affiliation as printed

   Carl-von-Ossietzky Universität, Oldenburg, Dept. Medical Physics and Acoustics, Cluster of Excellence “Hearing4all”, Carl-von-Ossietzky-Str. 9–11, 26129 Oldenburg, Germany

   Carl-von-Ossietzky Universität, Oldenburg, Dept. Medical Physics and Acoustics, Cluster of Excellence "Hearing4all", Carl-von-Ossietzky-Str. 9–11, 26129 Oldenburg, Germany

6. Josep Llorca-Bofí Aachen Institute for Hearing Technology and Acoustics

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Institute for Hearing Technology and Acoustics, Kopernikusstr. 5, 52074 Aachen, Germany

7. Giso Grimm

   Carl von Ossietzky Universität Oldenburg · Hearing4all

   Affiliation as printed

   Carl-von-Ossietzky Universität, Oldenburg, Dept. Medical Physics and Acoustics, Cluster of Excellence “Hearing4all”, Carl-von-Ossietzky-Str. 9–11, 26129 Oldenburg, Germany

   Carl-von-Ossietzky Universität, Oldenburg, Dept. Medical Physics and Acoustics, Cluster of Excellence "Hearing4all", Carl-von-Ossietzky-Str. 9–11, 26129 Oldenburg, Germany

8. Maartje M. E. Hendrikse

   Carl von Ossietzky Universität Oldenburg · Erasmus MC · Hearing4all

   Affiliation as printed

   Carl-von-Ossietzky Universität, Oldenburg, Dept. Medical Physics and Acoustics, Cluster of Excellence “Hearing4all”, Carl-von-Ossietzky-Str. 9–11, 26129 Oldenburg, Germany

   Erasmus University Medical Center, Rotterdam, Department of Otorhinolaryngology and Head and Neck Surgery, Burgemeester Oudlaan 50, 3062 PA Rotterdam, Netherlands

   Carl-von-Ossietzky Universität, Oldenburg, Dept. Medical Physics and Acoustics, Cluster of Excellence "Hearing4all", Carl-von-Ossietzky-Str. 9–11, 26129 Oldenburg, Germany

9. Birger Kollmeier

   Carl von Ossietzky Universität Oldenburg · Hearing4all

   Affiliation as printed

   Carl-von-Ossietzky Universität, Oldenburg, Dept. Medical Physics and Acoustics, Cluster of Excellence “Hearing4all”, Carl-von-Ossietzky-Str. 9–11, 26129 Oldenburg, Germany

   Carl-von-Ossietzky Universität, Oldenburg, Dept. Medical Physics and Acoustics, Cluster of Excellence "Hearing4all", Carl-von-Ossietzky-Str. 9–11, 26129 Oldenburg, Germany

10. Bernhard U. Seeber corresponding

    Technical University of Munich

    Affiliation as printed

    Technical University of Munich, Audio Information Processing, Department of Electrical and Computer Engineering, Theresienstr. 90, 80333 München, Germany