BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Video

IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 1674–1683

Abstract

Multiple existing benchmarks involve tracking and segmenting objects in video e.g., Video Object Segmentation (VOS) and Multi-Object Tracking and Segmentation (MOTS), but there is little interaction between them due to the use of disparate benchmark datasets and metrics (e.g. $\mathcal{J}\& {\mathcal{F}}$, mAP, sMOTSA). As a result, published works usually target a particular benchmark, and are not easily comparable to each another. We believe that the development of generalized methods that can tackle multiple tasks requires greater cohesion among these research sub-communities. In this paper, we aim to facilitate this by proposing BURST, a dataset which contains thousands of diverse videos with high-quality object masks, and an associated benchmark with six tasks involving object tracking and segmentation in video. All tasks are evaluated using the same data and comparable metrics, which enables researchers to consider them in unison, and hence, more effectively pool knowledge from different methods across different tasks. Additionally, we demonstrate several baselines for all tasks and show that approaches for one task can be applied to another with a quantifiable and explainable performance difference. Dataset annotations are available at: https://github.com/Ali2500/BURST-benchmark.

Authors 7

1. Ali Athar Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Germany

   RWTH Aachen University, Germany

2. Jonathon Luiten Aachen

   Carnegie Mellon University · RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Germany

   Carnegie Mellon University, USA

   RWTH Aachen University, Germany

3. Paul Voigtlaender

   Google (United States)

   Affiliation as printed

   Google

4. Tarasha Khurana

   Carnegie Mellon University

   Affiliation as printed

   Carnegie Mellon University,USA

   Carnegie Mellon University, USA

5. Achal Dave

   Amazon (United States)

   Affiliation as printed

   Amazon

6. Bastian Leibe Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Germany

   RWTH Aachen University, Germany

7. Deva Ramanan

   Carnegie Mellon University

   Affiliation as printed

   Carnegie Mellon University,USA

   Carnegie Mellon University, USA