Aspen Open Jets: unlocking LHC data for foundation models in particle physics

Machine Learning Science and Technology, vol. 6, pp. 030601

Abstract

Abstract Foundation models are deep learning models pre-trained on large amounts of data which are capable of generalizing to multiple datasets and/or downstream tasks. This work demonstrates how data collected by the CMS experiment at the Large Hadron Collider can be useful in pre-training foundation models for HEP. Specifically, we introduce the AspenOpenJets (AOJs) dataset, consisting of approximately 178 M high p T jets derived from CMS 2016 Open Data. We show how pre-training the OmniJet - α foundation model on AOJs improves performance on generative tasks with significant domain shift: generating boosted top and QCD jets from the simulated JetClass dataset. In addition to demonstrating the power of pre-training of a jet-based foundation model on actual proton–proton collision data, we provide the ML-ready derived AOJs dataset for further public use.

Authors 9

1. Oz Amram

   Fermi National Accelerator Laboratory

   Affiliation as printed

   Fermi National Accelerator Laboratory, Batavia, IL 60510, United States of America

2. Luca Anzalone

   Istituto Nazionale di Fisica Nucleare · Istituto Nazionale di Fisica Nucleare, Sezione di Bologna · University of Bologna

   Affiliation as printed

   Department of Physics and Astronomy (DIFA), University of Bologna, 40127 Bologna, Italy

   Istituto Nazionale di Fisica Nucleare (INFN), 40127 Bologna, Italy

3. Joschka Birk

   Universität Hamburg

   Affiliation as printed

   Institut für Experimentalphysik, Universität Hamburg, 22761 Hamburg, Germany

4. Darius A. Faroughy

   Rutgers, The State University of New Jersey

   Affiliation as printed

   NHETC, Dept. of Physics and Astronomy, Rutgers University, Piscataway, NJ 08854, United States of America

5. Anna Hallin

   Universität Hamburg

   Affiliation as printed

   Institut für Experimentalphysik, Universität Hamburg, 22761 Hamburg, Germany

6. Gregor Kasieczka

   Universität Hamburg

   Affiliation as printed

   Center for Data and Computing in Natural Sciences (CDCS), 22607 Hamburg, Germany

   Institut für Experimentalphysik, Universität Hamburg, 22761 Hamburg, Germany

7. Michael Krämer Aachen Institut für Theoretische Teilchenphysik und Kosmologie

   RWTH Aachen University

   Affiliation as printed

   Institut für Theoretische Teilchenphysik und Kosmologie, RWTH Aachen University, 52074 Aachen, Germany

8. Ian Pang

   Rutgers, The State University of New Jersey

   Affiliation as printed

   NHETC, Dept. of Physics and Astronomy, Rutgers University, Piscataway, NJ 08854, United States of America

9. Humberto Reyes-González Aachen Institut für Theoretische Teilchenphysik und Kosmologie

   RWTH Aachen University

   Affiliation as printed

   Institut für Theoretische Teilchenphysik und Kosmologie, RWTH Aachen University, 52074 Aachen, Germany