Movement Sonification Types and Triggers: A Systematic Review

Perceptual and Motor Skills, pp. 315125261448460

Abstract

Movement sonification (i.e., translating kinematic measures into sound, providing auditory feedback), can enhance motor performance and (re)learning across both clinical and sports contexts. However, existing sonification systems vary widely, potentially affecting efficacy. Here, we categorize sonification systems based on the utilized sound (type) and onset (trigger) to elucidate potential differences in movement-, intuitiveness-, and motivation-related outcomes. Movement sonification systems, as described in 101 included studies, were classified into six sound types (specifically parameter mapping, musically informed, single tone, rhythmic, music-based, and environmental) and four trigger types (event-, error-, trajectory-based, and continuous), and effects on outcomes were compared. Overall, positive effects on all outcomes were reported for all system types and triggers, with no clear pattern of advantages for any, except that learning retention is more consistently reported for triggers involving explicit learning strategies. In general, more subtle movement learning may benefit from more direct movement-sound coupling, implicit learning, and multi-component mappings. Additionally, systems providing rhythmic sounds (musically informed, single tone, rhythm, music-based) were more commonly used for periodic movements. Intuitiveness is rated highest for parameter mapping and rhythmic sonification, and motivation strongest for musically informed sonification, although all systems increase autonomous motivation, compared to no sonification. In conclusion, movement sonification systems generally improve performance but arguably work differently for different movements. Accordingly, musically informed sonification may be preferable for rehabilitation settings where motivation is crucial, while parameter mapping sonification that inform users about subtle kinematic aspects may especially support learning highly skilled movements. Standardizing terminology in future work will facilitate systematic cross-system comparisons.

Authors 5

1. Marijn Coers corresponding Aachen

   Leiden University

   Affiliation as printed

   Leiden University

2. Maria M. Petratza Aachen

   Leiden University

   Affiliation as printed

   Leiden University

3. Marit F. L. Ruitenberg Aachen

   Leiden University

   Affiliation as printed

   Leiden University

4. Ineke J.M. van der Ham Aachen

   Leiden University · Delft University of Technology

   Affiliation as printed

   Faculty of Architecture and the Built Environment, Delft University of Technology, Delft, The Netherlands

   Leiden University

5. Rebecca S. Schaefer Aachen

   Leiden University

   Affiliation as printed

   Leiden University