How to pick the best anomaly detector?

Physical review. D/Physical review. D., vol. 114

Abstract

Anomaly detection has the potential to discover new physics in unexplored regions of the data. However, choosing the best anomaly detector for a given dataset in a model-agnostic way is an important challenge which has hitherto largely been neglected. In this paper, we introduce the data-driven ARGOS metric, which has a sound theoretical foundation and is empirically shown to robustly select the most sensitive anomaly detection model given the data. Focusing on weakly supervised, classifier-based anomaly detection methods, we show that the ARGOS metric outperforms other model selection metrics previously used in the literature, in particular the binary cross-entropy loss. We explore several realistic applications, including hyperparameter tuning as well as architecture and feature selection. In all cases, we demonstrate that ARGOS is highly sensitive to the small amounts of signal present in anomaly detection analyses while at the same time being robust to background overfitting and background template mismodeling.

Authors 6

1. Marie Hein Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University

2. Gregor Kasieczka

   Universität Hamburg

   Affiliation as printed

   Universität Hamburg

3. Michael Krämer Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University

4. L. Moureaux

   Universität Hamburg

   Affiliation as printed

   Universität Hamburg

5. Alexander Mück Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University

6. David Shih

   Rutgers, The State University of New Jersey

   Affiliation as printed

   Rutgers University