Refining weakly supervised anomaly detection for new physics searches

RWTH Publications (RWTH Aachen)

Abstract

Since the discovery of the Higgs boson, numerous searches for physics beyond the Standard Model have been conducted at the Large Hadron Collider, so far without success. This has motivated the development of novel, model-agnostic search strategies. In particular, machine learning has allowed for the development of highly sensitive anomaly detection methods. Among these, weakly supervised anomaly detection has shown great promise as it provides a well-defined, controllable anomaly score by training a classifier on two mixed data sets: a Signal Region (SR), which may contain background and signal, and a background-only Background Template (BT). In this thesis, we focus on improving both the performance and our understanding of weakly supervised anomaly detection. We explore two parts of a weakly supervised search in detail, namely the classifier and how significances are obtained. First, we replace the Multi-Layer Perceptron (MLP), which was used in early proof-of-concept studies, with a Gradient-Boosted Decision Tree (GBDT). The GBDT performs well for the high-level feature sets typically used in weakly supervised anomaly detection and limits the classifier's susceptibility to irrelevant features in the classification feature set. Second, we show that optimizing an anomaly detection analysis in a data-driven manner is possible. For this, we explore a number of optimization tasks, namely the selection of the best epochs, hyperparameter optimization, architecture choice, and feature selection. Such data-driven choices are made possible by the introduction of the new ARGOS (Above Random Gain of SIC) metric, which is strongly correlated with the significance improvement of a classifier. We also introduce a new method to obtain the significance of a potential discovery in a weakly supervised analysis. Previous weakly supervised resonant analyses have relied on fits to the background mass spectrum. This has made them vulnerable to the sculpting of the resonant mass spectrum. Our "direct background estimation" instead simply compares the selected event numbers in the BT and SR after a cut on the anomaly detector, thereby avoiding a fit. We account for systematic biases from imperfect BTs using both simulation and data-driven methods. Lastly, we explore the statistical behavior of anomaly detectors to ensure that significances are calibrated correctly. In particular, we investigate whether a look-elsewhere effect is incurred by anomaly detectors trained to select events from a large phase space. By analyzing miscalibrations of p-values and their effect on sensitivity, we provide practical guidance for future analyses.

Authors 1

1. Marie Hein corresponding Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen