Multilevel rhythms in multimodal communication

Philosophical Transactions of the Royal Society B Biological Sciences, vol. 376, pp. 20200334

Abstract

It is now widely accepted that the brunt of animal communication is conducted via several modalities, e.g. acoustic and visual, either simultaneously or sequentially. This is a laudable multimodal turn relative to traditional accounts of temporal aspects of animal communication which have focused on a single modality at a time. However, the fields that are currently contributing to the study of multimodal communication are highly varied, and still largely disconnected given their sole focus on a particular level of description or their particular concern with human or non-human animals. Here, we provide an integrative overview of converging findings that show how multimodal processes occurring at neural, bodily, as well as social interactional levels each contribute uniquely to the complex rhythms that characterize communication in human and non-human animals. Though we address findings for each of these levels independently, we conclude that the most important challenge in this field is to identify how processes at these different levels connect. This article is part of the theme issue 'Synchrony and rhythm interaction: from the brain to behavioural ecology'.

Authors 8

1. Wim T. J. L. Pouw corresponding

   Radboud University Nijmegen · Max Planck Institute for Psycholinguistics · Donders Institute for Brain, Cognition and Behaviour

   Affiliation as printed

   Donders Institute for Brain, Cognition and Behaviour 1 , ,

   Max Planck Institute for Psycholinguistics 2 ,

   Nijmegen, The Netherlands 1 , ,

   Nijmegen, The Netherlands 2 ,

   Radboud University 1 , ,

   Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands

   Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands

2. Shannon Proksch

   University of California, Merced

   Affiliation as printed

   Cognitive and Information Sciences 3 , ,

   Merced, CA, USA 3 , ,

   University of California 3 , ,

   Cognitive and Information Sciences, University of California, Merced, CA, USA

3. Linda Drijvers

   Radboud University Nijmegen · Max Planck Institute for Psycholinguistics · Donders Institute for Brain, Cognition and Behaviour

   Affiliation as printed

   Donders Institute for Brain, Cognition and Behaviour 1 , ,

   Max Planck Institute for Psycholinguistics 2 ,

   Nijmegen, The Netherlands 1 , ,

   Nijmegen, The Netherlands 2 ,

   Radboud University 1 , ,

   Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands

   Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands

4. Marco Gamba

   University of Turin

   Affiliation as printed

   Department of Life Sciences and Systems Biology 4 , ,

   Turin, Italy 4 , ,

   University of Turin 4 , ,

   Department of Life Sciences and Systems Biology, University of Turin, Turin, Italy

5. Judith Holler

   Radboud University Nijmegen · Max Planck Institute for Psycholinguistics · Donders Institute for Brain, Cognition and Behaviour

   Affiliation as printed

   Donders Institute for Brain, Cognition and Behaviour 1 , ,

   Max Planck Institute for Psycholinguistics 2 ,

   Nijmegen, The Netherlands 1 , ,

   Nijmegen, The Netherlands 2 ,

   Radboud University 1 , ,

   Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands

   Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands

6. Christopher T. Kello

   University of California, Merced

   Affiliation as printed

   Cognitive and Information Sciences 3 , ,

   Merced, CA, USA 3 , ,

   University of California 3 , ,

   Cognitive and Information Sciences, University of California, Merced, CA, USA

7. Rebecca S. Schaefer Aachen Health Medical and Neuropsychology Unit Institute for Psychology, Leiden University

   Leiden University

   Affiliation as printed

   Academy for Creative and Performing Arts 6 , ,

   Health, Medical and Neuropsychology unit, Institute for Psychology 5 , ,

   Leiden University 5 , ,

   Leiden University 6 , ,

   Leiden, The Netherlands 5 , ,

   Leiden, The Netherlands 6 , ,

   Academy for Creative and Performing Arts, Leiden University, Leiden, The Netherlands

   Health, Medical and Neuropsychology unit, Institute for Psychology, Leiden University, Leiden, The Netherlands

8. Geraínt A. Wiggins

   Vrije Universiteit Brussel · Queen Mary University of London

   Affiliation as printed

   Brussels, Belgium and Queen Mary University of London, UK 7 ,

   London, UK 8 ,

   Queen Mary University 8 ,

   Vrije Universiteit Brussel 7 ,

   Queen Mary University, London, UK

   Vrije Universiteit Brussel, Brussels, Belgium and Queen Mary University of London, UK