PATH: A discrete-sequence dataset for evaluating online unsupervised anomaly detection approaches for multivariate time series

Big Data Research, vol. 42, pp. 100573

Abstract

Benchmarking anomaly detection approaches for multivariate time series is a challenging task due to a lack of high-quality datasets. Current publicly available datasets are too small, not diverse and feature trivial anomalies, which hinders measurable progress in this research area. We propose a solution: a diverse, extensive, and non-trivial dataset generated via state-of-the-art simulation tools that reflects realistic behaviour of an automotive powertrain, including its multivariate, dynamic and variable-state properties. Additionally, our dataset represents a discrete-sequence problem, which remains unaddressed by previously-proposed solutions in literature. To cater for both unsupervised and semi-supervised anomaly detection settings, as well as time series generation and forecasting, we make different versions of the dataset available, where training and test subsets are offered in contaminated and clean versions, depending on the task. We also provide baseline results from a selection of approaches based on deterministic and variational autoencoders, as well as a non-parametric approach. As expected, the baseline experimentation shows that the approaches trained on the semi-supervised version of the dataset outperform their unsupervised counterparts, highlighting a need for approaches more robust to contaminated training data. Furthermore, results show that the threshold used can have a large influence on detection performance, hence more work needs to be invested in methods to find a suitable threshold without the need for labelled data.

Authors 4

1. Lucas Correia corresponding Aachen

   Leiden University · Mercedes-Benz (Germany)

   Affiliation as printed

   LIACS, Leiden University, Einsteinweg 55, 2333 CC, Leiden, the Netherlands

   Mercedes-Benz AG, Mercedesstraße 120, 70372, Stuttgart, Germany

2. Jan-Christoph Goos

   Mercedes-Benz (Germany)

   Affiliation as printed

   Mercedes-Benz AG, Mercedesstraße 120, 70372, Stuttgart, Germany

3. Thomas Bäck Aachen

   Leiden University

   Affiliation as printed

   LIACS, Leiden University, Einsteinweg 55, 2333 CC, Leiden, the Netherlands

4. Anna V. Kononova Aachen

   Leiden University

   Affiliation as printed

   LIACS, Leiden University, Einsteinweg 55, 2333 CC, Leiden, the Netherlands