Phase-Based Data Placement Optimization in Heterogeneous Memory

Proceedings, pp. 382–393

Abstract

While scientific applications show increasing demand for memory speed and capacity, the performance gap between compute cores and the memory subsystem continues to spread. In response, heterogeneous memory systems integrating high-bandwidth memory (HBM) and non-volatile memory (NVM) alongside traditional DRAM on the CPU side are gaining traction. Despite the potential benefits of optimized memory selection for improved performance and efficiency, adapting applications to leverage diverse memory types often requires extensive modifications. Moreover, applications often comprise multiple execution phases with varying data access patterns. Since the capacity of the “fastest” memory is limited, relying solely on fixed data placement decisions may not yield optimal performance. Thus, considering allocation lifetimes and dynamically migrating data between memory types becomes imperative to ensure that performance-critical data for each phase resides in fast memory. To address these challenges, we developed a workflow incorporating memory access profiling, optimization techniques and a runtime system, which selects initial data placement for allocations and performs data migration during execution, considering the platform's memory subsystem characteristics and capacities. We formalize the optimization problems for initial and phase-based data placement and propose heuristics derived from memory profiling metrics to solve it. Additionally, we outline the implementation of these approaches, including allocation interception to enforce placement decisions. Experiments conducted with several applications on an Intel Ice Lake$(\text{DRAM}+\text{NVM})$and Sapphire Rapids$(\text{HBM}+\text{DRAM})$system demonstrate that our methodology can effectively bridge the performance gap between slow and fast memory in heterogeneous memory environments.

Authors 6

1. Jannis Klinkenberg Aachen Chair for High Performance Computing

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Chair for High Performance Computing,Aachen,Germany

2. Clément Foyer

   Commissariat à l'Énergie Atomique et aux Énergies Alternatives · Laboratoire d'Informatique en Calcul Intensif et Image pour la Simulation · Université de Reims Champagne-Ardenne

   Affiliation as printed

   Université de Reims Champagne-Ardenne,CEA, LRC DIGIT, LICIIS,Reims,France

3. Brice Goglin

   Institut national de recherche en sciences et technologies du numérique · Université de Bordeaux · Laboratoire Bordelais de Recherche en Informatique

   Affiliation as printed

   Inria, Univ. Bordeaux, LaBRI,Talence,France

4. Emmanuel Jeannot

   Institut national de recherche en sciences et technologies du numérique · Université de Bordeaux · Laboratoire Bordelais de Recherche en Informatique

   Affiliation as printed

   Inria, Univ. Bordeaux, LaBRI,Talence,France

5. Christian Terboven Aachen Chair for High Performance Computing

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Chair for High Performance Computing,Aachen,Germany

6. Anara Kozhokanova Aachen Chair for High Performance Computing

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Chair for High Performance Computing,Aachen,Germany