Multilevel rhythms in multimodal communication

Abstract

It is now widely accepted that the brunt of animal communication is conducted via several modalities, e.g. acoustic and visual, either simultaneously or sequentially. This is a laudable multimodal turn relative to traditional accounts of temporal aspects of animal communication which have focused on a single modality at a time. However, the fields that are currently contributing to the study of multimodal communication are highly varied, and still largely disconnected given their sole focus on a particular level of description or their particular concern with human or non-human animals. Here we provide an integrative overview of converging findings that show how multimodal processes occurring at neural, bodily, as well as social interactional levels each contribute uniquely to the complex rhythms that characterize communication in human and non-human animals. Though we address findings for each of these levels independently, we conclude that the most important challenge in this field is to identify how processes at these different levels connect.

Authors 8

1. Wim T. J. L. Pouw corresponding

   Radboud University Nijmegen · Max Planck Institute for Psycholinguistics · Donders Institute for Brain, Cognition and Behaviour

   Affiliation as printed

   . Donders Institute for Cognition, Brain and Behaviour, Radboud University, Nijmegen, The Netherlands

   . Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands

2. Shannon Proksch

   University of California, Merced

   Affiliation as printed

   . Cognitive and Information Sciences, University of California, Merced, USA

3. Linda Drijvers

   Radboud University Nijmegen · Max Planck Institute for Psycholinguistics · Donders Institute for Brain, Cognition and Behaviour

   Affiliation as printed

   . Donders Institute for Cognition, Brain and Behaviour, Radboud University, Nijmegen, The Netherlands

   . Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands

4. Marco Gamba

   University of Turin

   Affiliation as printed

   . Department of Life Sciences and Systems Biology, University of Turin, Italy

5. Judith Holler

   Radboud University Nijmegen · Max Planck Institute for Psycholinguistics · Donders Institute for Brain, Cognition and Behaviour

   Affiliation as printed

   . Donders Institute for Cognition, Brain and Behaviour, Radboud University, Nijmegen, The Netherlands

   . Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands

6. Christopher T. Kello

   University of California, Merced

   Affiliation as printed

   . Cognitive and Information Sciences, University of California, Merced, USA

7. Rebecca S. Schaefer Aachen Health Medical and Neuropsychology Unit Institute for Psychology, Leiden University

   Leiden University

   Affiliation as printed

   . Academy for Creative and Performing Arts, Leiden University, Leiden, The Netherlands

   . Health, Medical and Neuropsychology unit, Institute for Psychology, Leiden University, Leiden, The Netherlands

8. Geraínt A. Wiggins

   Vrije Universiteit Brussel · Queen Mary University of London

   Affiliation as printed

   . Vrije Universiteit Brussel, Belgium & Queen Mary University of London, UK