Fast Columnar Physics Analyses of Terabyte-Scale LHC Data on a Cache-Aware Dask Cluster

Computing and Software for Big Science, vol. 7

Abstract

Abstract The development of an LHC physics analysis involves numerous investigations that require the repeated processing of terabytes of data. Thus, a rapid completion of each of these analysis cycles is central to mastering the science project. We present a solution to efficiently handle and accelerate physics analyses on small-size institute clusters. Our solution uses three key concepts: vectorized processing of collision events, the “MapReduce” paradigm for scaling out on computing clusters, and efficiently utilized SSD caching to reduce latencies in IO operations. This work focuses on the latter key concept, its underlying mechanism, and its implementation. Using simulations from a Higgs pair production physics analysis as an example, we achieve an improvement factor of 6.3 in the runtime for reading all input data after one cycle and even an overall speedup of a factor of 14.9 after 10 cycles, reducing the runtime from hours to minutes.

Authors 6

1. N. Eich Aachen Physics Institute 3A

   RWTH Aachen University

   Affiliation as printed

   Physics Institute 3A, RWTH Aachen University, 52056, Aachen, Germany

2. Martin Erdmann Aachen Physics Institute 3A

   RWTH Aachen University

   Affiliation as printed

   Physics Institute 3A, RWTH Aachen University, 52056, Aachen, Germany

3. P. Fackeldey corresponding Aachen Physics Institute 3A

   RWTH Aachen University

   Affiliation as printed

   Physics Institute 3A, RWTH Aachen University, 52056, Aachen, Germany

4. Benjamin Fischer Aachen Physics Institute 3A

   RWTH Aachen University

   Affiliation as printed

   Physics Institute 3A, RWTH Aachen University, 52056, Aachen, Germany

5. D. Noll Aachen Physics Institute 3A

   RWTH Aachen University

   Affiliation as printed

   Physics Institute 3A, RWTH Aachen University, 52056, Aachen, Germany

6. Yannik Alexander Rath Aachen Physics Institute 3A

   RWTH Aachen University

   Affiliation as printed

   Physics Institute 3A, RWTH Aachen University, 52056, Aachen, Germany