Learning the language of QCD jets with transformers

Journal of High Energy Physics, vol. 2023

Abstract

Abstract Transformers have become the primary architecture for natural language processing. In this study, we explore their use for auto-regressive density estimation in high-energy jet physics, which involves working with a high-dimensional space. We draw an analogy between sentences and words in natural language and jets and their constituents in high-energy physics. Specifically, we investigate density estimation for light QCD jets and hadronically decaying boosted top jets. Since transformers allow easy sampling from learned densities, we exploit their generative capability to assess the quality of the density estimate. Our results indicate that the generated data samples closely resemble the original data, as evidenced by the excellent agreement of distributions such as particle multiplicity or jet mass. Furthermore, the generated samples are difficult to distinguish from the original data, even by a powerful supervised classifier. Given their exceptional data processing capabilities, transformers could potentially be trained directly on the massive LHC data sets to learn the probability densities in high-energy jet physics.

Authors 4

1. Thorben Finke corresponding Aachen Institute for Theoretical Particle Physics and Cosmology

   RWTH Aachen University

   Affiliation as printed

   Institute for Theoretical Particle Physics and Cosmology (TTK), RWTH Aachen University, D-52056, Aachen, Germany

2. M. Krämer Aachen Institute for Theoretical Particle Physics and Cosmology

   RWTH Aachen University

   Affiliation as printed

   Institute for Theoretical Particle Physics and Cosmology (TTK), RWTH Aachen University, D-52056, Aachen, Germany

3. Alexander Mück Aachen Institute for Theoretical Particle Physics and Cosmology

   RWTH Aachen University

   Affiliation as printed

   Institute for Theoretical Particle Physics and Cosmology (TTK), RWTH Aachen University, D-52056, Aachen, Germany

4. Jan Tönshoff Aachen Chair of Computer Science 7 (Logic and Theory of Discrete Systems) Department of Computer Science

   RWTH Aachen University

   Affiliation as printed

   Chair of Computer Science 7 (Logic and Theory of Discrete Systems), Department of Computer Science, RWTH Aachen University, Aachen, Germany