Learning to detect an animal sound from five examples

Ecological Informatics, vol. 77, pp. 102258

Abstract

Automatic detection and classification of animal sounds has many applications in biodiversity monitoring and animal behavior. In the past twenty years, the volume of digitised wildlife sound available has massively increased, and automatic classification through deep learning now shows strong results. However, bioacoustics is not a single task but a vast range of small-scale tasks (such as individual ID, call type, emotional indication) with wide variety in data characteristics, and most bioacoustic tasks do not come with strongly-labelled training data. The standard paradigm of supervised learning, focussed on a single large-scale dataset and/or a generic pre-trained algorithm, is insufficient. In this work we recast bioacoustic sound event detection within the AI framework of few-shot learning. We adapt this framework to sound event detection, such that a system can be given the annotated start/end times of as few as 5 events, and can then detect events in long-duration audio—even when the sound category was not known at the time of algorithm training. We introduce a collection of open datasets designed to strongly test a system's ability to perform few-shot sound event detections, and we present the results of a public contest to address the task. Our analysis shows that prototypical networks are a very common used strategy and they perform well when enhanced with adaptations for general characteristics of animal sounds. However, systems with high time resolution capabilities perform the best in this challenge. We demonstrate that widely-varying sound event durations are an important factor in performance, as well as non-stationarity, i.e. gradual changes in conditions throughout the duration of a recording. For fine-grained bioacoustic recognition tasks without massive annotated training data, our analysis demonstrate that few-shot sound event detection is a powerful new method, strongly outperforming traditional signal-processing detection methods in the fully automated scenario.

Authors 18

1. Inês Nolasco

   Centre National de la Recherche Scientifique · Queen Mary University of London

   Affiliation as printed

   Centre National de la Recherche Scientifique (CNRS), France

   Queen Mary University of London, London, UK

2. Shubhr Singh

   Queen Mary University of London

   Affiliation as printed

   Queen Mary University of London, London, UK

3. Veronica Morfi

   Queen Mary University of London

   Affiliation as printed

   Queen Mary University of London, London, UK

4. Vincent Lostanlen

   École Centrale de Nantes · Centre National de la Recherche Scientifique · Laboratoire des Sciences du Numérique de Nantes · Nantes Université

   Affiliation as printed

   Nantes Universite, Ecole Centrale Nantes, CNRS, LS2N, UMR 6004, F-44000 Nantes, France

5. Ariana Strandburg‐Peshkin

   University of Konstanz · Max Planck Institute of Animal Behavior

   Affiliation as printed

   Biology Department, University of Konstanz, Universitätsstrasse 10, Konstanz, Germany

   Centre for the Advanced Study of Collective Behaviour, University of Konstanz, Universitätsstrasse 10, Konstanz, Germany

   Department for the Ecology of Animal Societies, Max Planck Institute of Animal Behavior, Bücklestrasse 5, Konstanz, Germany

6. Ester Vidaña-Vila

   Universitat Ramon Llull

   Affiliation as printed

   La Salle Campus Barcelona, Universitat Ramon Llull, Sant Joan de La Salle, 42, Barcelona, Spain

7. Lisa Gill
   Affiliation as printed

   Landesbund fur Vogel- und Naturschutz, Hilpoltstein, Germany

   Naturkundemuseum Bayern/BIOTOPIA Lab, Munich, Germany

8. Hanna Pamuła

   AGH University of Krakow

   Affiliation as printed

   AGH University of Science and Technology, al. Adama Mickiewicza 30, Krakow, Poland

9. Helen Whitehead

   University of Salford

   Affiliation as printed

   School of Science, Engineering and Environment,University of Salford, Manchester, UK

10. Ivan Kiskin

    University of Surrey

    Affiliation as printed

    Centre for Vision, Speech and Signal Processing, FEPS, University of Surrey, Surrey, UK

11. Frants Havmand Jensen

    Aarhus University · Woods Hole Oceanographic Institution · Syracuse University

    Affiliation as printed

    Aarhus University, Department of Ecoscience, Frederiksborgvej 399, 4000 Roskilde, Denmark

    Syracuse University, 107 College Place, Syracuse, NY 13244, USA

    Woods Hole Oceanographic Institution, 266 Woods Hole Rd, Woods Hole, MA 02543, USA

12. Joe Morford

    University of Oxford

    Affiliation as printed

    The Oxford Navigation group, Dept. of Zoology, University of Oxford, Oxford, UK

13. Michael G. Emmerson

    Queen Mary University of London

    Affiliation as printed

    Queen Mary University of London, London, UK

14. Elisabetta Versace

    Queen Mary University of London

    Affiliation as printed

    Queen Mary University of London, London, UK

15. Emily M. Grout

    University of Konstanz · Max Planck Institute of Animal Behavior

    Affiliation as printed

    Biology Department, University of Konstanz, Universitätsstrasse 10, Konstanz, Germany

    Centre for the Advanced Study of Collective Behaviour, University of Konstanz, Universitätsstrasse 10, Konstanz, Germany

    Department for the Ecology of Animal Societies, Max Planck Institute of Animal Behavior, Bücklestrasse 5, Konstanz, Germany

16. Haohe Liu

    University of Surrey

    Affiliation as printed

    Centre for Vision, Speech and Signal Processing, FEPS, University of Surrey, Surrey, UK

17. Burooj Ghani Aachen Naturalis Biodiversity Center

    Naturalis Biodiversity Center

    Affiliation as printed

    Naturalis Biodiversity Center, Leiden, the Netherlands

18. Dan Stowell corresponding Aachen Naturalis Biodiversity Center

    Naturalis Biodiversity Center · Tilburg University

    Affiliation as printed

    Naturalis Biodiversity Center, Leiden, the Netherlands

    Tilburg University, Tilburg, the Netherlands