Bird song comparison using deep learning trained from avian perceptual judgments

bioRxiv (Cold Spring Harbor Laboratory)

Abstract

Abstract Our understanding of bird song, a model system for animal communication and the neurobiology of learning, depends critically on making reliable, validated comparisons between the complex multidimensional syllables that are used in songs. However, most assessments of song similarity are based on human inspection of spectrograms, or computational methods developed from human intuitions. Using a novel automated operant conditioning system, we collected a large corpus of zebra finches’ ( Taeniopygia guttata ) decisions about song syllable similarity. We use this dataset to compare and externally validate similarity algorithms in widely-used publicly available software (Raven, Sound Analysis Pro, Luscinia). Although these methods all perform better than chance, they do not closely emulate the avian assessments. We then introduce a novel deep learning method that can produce perceptual similarity judgements trained on such avian decisions. We find that this new method outperforms the established methods in accuracy and more closely approaches the avian assessments. Inconsistent (hence ambiguous) decisions are a common occurrence in animal behavioural data; we show that a modification of the deep learning training that accommodates these leads to the strongest performance. We argue this approach is the best way to validate methods to compare song similarity, that our dataset can be used to validate novel methods, and that the general approach can easily be extended to other species.

Authors 6

1. Lies Zandberg

   Queen Mary University of London · Royal Holloway University of London

   Affiliation as printed

   Department of Psychology, Queen Mary University of London, UK

   Department of Psychology, Royal Holloway University of London, UK

   Deprtment of Psychology, Royl Hollowy University of London, UK

2. Veronica Morfi

   Queen Mary University of London

   Affiliation as printed

   Machine Listening Lab, Centre for Digital Music (C4DM), Queen Mary University of London, UK

   Mahine Listening Lab, Centre for Digital Musi (C4DM), Queen Mary University of London, UK

3. Julia M. George

   Queen Mary University of London · Clemson University

   Affiliation as printed

   Department of Biological Sciences, Clemson University, Clemson, SC, USA

   Department of Psychology, Queen Mary University of London, UK

4. David F. Clayton

   Queen Mary University of London · Clemson University

   Affiliation as printed

   Department of Genetics and Biochemistry, Clemson University, Clemson, SC, USA

   Department of Psychology, Queen Mary University of London, UK

   Dpartmnt of Gntics and Biochmistry, Clmson Univrsity, Clmson, SC, USA

5. Dan Stowell Aachen Naturalis Biodiversity Center

   Naturalis Biodiversity Center · Queen Mary University of London · Tilburg University

   Affiliation as printed

   Department of Cognitive Science and AI, Tilburg University, Netherlands

   Machine Listening Lab, Centre for Digital Music (C4DM), Queen Mary University of London, UK

   Naturalis Biodiversity Centre, Leiden, Netherlands

   Department o Cognitive Science and AI, Tilburg University, Netherlands

   Mahine Listening Lab, Centre for Digital Musi (C4DM), Queen Mary University of London, UK

6. Robert F. Lachlan corresponding

   Queen Mary University of London · Royal Holloway University of London

   Affiliation as printed

   Department of Psychology, Queen Mary University of London, UK

   Department of Psychology, Royal Holloway University of London, UK

   Deprtment of Psychology, Royl Hollowy University of London, UK