An Experimental Setup to Evaluate RAPL Energy Counters for Heterogeneous Memory

ACM/SPEC International Conference on Performance Engineering (ICPE), pp. 71–82

Abstract

Power consumption of the main memory in modern heterogeneous high-performance computing (HPC) constitutes a significant part of the total power consumption of a node. This motivates energy-efficient solutions targeting the memory domain as well. Practitioners need reliable energy measurement techniques for analyzing energy and power consumption of applications and performance optimizations. Running Average Power Limit (RAPL) is a common choice, as it provides uncomplicated access to the energy measurements. While RAPL's accuracy has been studied and validated on homogeneous memory platforms, no work we are aware of investigated its accuracy on heterogeneous memory platforms, specifically with high-capacity memory (HCM). This paper describes the process of measuring the memory power consumption externally using riser cards in detail. We validate RAPL's accuracy by comparing results obtained from Intel's Ice Lake-SP system equipped with DDR4 DRAM and Intel Optane Persistent Memory Modules (PMM). In addition, we verify the accuracy of our instrumentation setup by comparing the results from an older Broadwell system with the results in the literature. We show that the RAPL values on a heterogeneous memory system report a higher offset from the reference measurements. The difference is more pronounced at lower memory load for all memory types. Also, we find that RAPL readings are inconsistent between multiple sockets and over time. Based on the evaluated scenarios, we conclude that RAPL overestimates the actual power consumption on heterogeneous memory systems and provide a discussion on the possible causes of this effect.

Authors 5

1. Lukas Alt Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Aachen, Germany

2. Anara Kozhokanova Aachen IT centre

   RWTH Aachen University

   Affiliation as printed

   IT Center, RWTH Aachen University, Aachen, Germany

3. Thomas Ilsche

   Center for Interdisciplinary Digital Sciences · Center for Information Services and High Performance Computing · Technische Universität Dresden

   Affiliation as printed

   CIDS, Information Services and High Performance Computing (ZIH), TU Dresden, Dresden, Germany

4. Christian Terboven Aachen IT centre

   RWTH Aachen University

   Affiliation as printed

   IT Center, RWTH Aachen University, Aachen, Germany

5. Matthias Mueller Aachen IT centre

   RWTH Aachen University

   Affiliation as printed

   IT Center, RWTH Aachen University, Aachen, Germany