Fast Columnar Physics Analyses of Terabyte-Scale LHC Data on a Cache-Aware Dask Cluster

arXiv (Cornell University)

Abstract

The development of an LHC physics analysis involves numerous investigations that require the repeated processing of terabytes of data. Thus, a rapid completion of each of these analysis cycles is central to mastering the science project. We present a solution to efficiently handle and accelerate physics analyses on small-size institute clusters. Our solution is based on three key concepts: Vectorized processing of collision events, the "MapReduce" paradigm for scaling out on computing clusters, and efficiently utilized SSD caching to reduce latencies in IO operations. Using simulations from a Higgs pair production physics analysis as an example, we achieve an improvement factor of $6.3$ in runtime after one cycle and even an overall speedup of a factor of $14.9$ after $10$ cycles.

Authors 5

1. N. Eich Aachen Physics Institute 3A

   RWTH Aachen University

   Affiliation as printed

   Physics Institute 3A , RWTH Aachen University , 52056 Aachen , Germany

2. Martin Erdmann Aachen Physics Institute 3A

   RWTH Aachen University

   Affiliation as printed

   Physics Institute 3A , RWTH Aachen University , 52056 Aachen , Germany

3. P. Fackeldey Aachen Physics Institute 3A

   RWTH Aachen University

   Affiliation as printed

   Physics Institute 3A , RWTH Aachen University , 52056 Aachen , Germany

4. D. Noll Aachen Physics Institute 3A

   RWTH Aachen University

   Affiliation as printed

   Physics Institute 3A , RWTH Aachen University , 52056 Aachen , Germany

5. Yannik Alexander Rath Aachen Physics Institute 3A

   RWTH Aachen University

   Affiliation as printed

   Physics Institute 3A , RWTH Aachen University , 52056 Aachen , Germany