H2M: Exploiting Heterogeneous Shared Memory Architectures

Future Generation Computer Systems, vol. 148, pp. 39–55

Abstract

Over the past decades, the performance gap between the memory subsystem and compute capabilities continued to spread. However, scientific applications and simulations show increasing demand for both memory speed and capacity. To tackle these demands, new technologies such as high-bandwidth memory (HBM) or non-volatile memory (NVM) emerged, which are usually combined with classical DRAM. The resulting architecture is a heterogeneous memory system in which no single memory is “best”. HBM is smaller but offers higher bandwidth than DRAM, whereas NVM provides larger capacity than DRAM at a reasonable cost and less energy consumption. Despite that, in several cases, DRAM still offers the best latency out of all three technologies. In order to use different kinds of memory, applications typically have to be modified to a great extent. Consequently, vendor-agnostic solutions are desirable. First, they should offer the functionality to identify kinds of memory, and second, to allocate data on it. In addition, because memory capacities may be limited, decisions about data placement regarding the different memory kinds have to be made. Finally, in making these decisions, changes over time in data that is accessed, and the actual access pattern, should be considered for initial data placement and be respected in data migration at run-time. In this paper, we introduce a new methodology that aims to provide portable tools and methods for managing data placement in systems with heterogeneous memory. Our approach allows programmers to provide traits (hints) for allocations that describe how data is used and accessed. Combined with characteristics of the platforms’ memory subsystem, these traits are exploited by heuristics to decide where to place data items. We also discuss methodologies for analyzing and identifying memory access characteristics of existing applications, and for recommending allocation traits. In our evaluation, we conduct experiments with several kernels and two proxy applications on Intel Knights Landing (HBM + DRAM) and Intel Ice Lake with Intel Optane DC Persistent Memory (DRAM + NVM) systems. We demonstrate that our methodology can bridge the performance gap between slow and fast memory by applying heuristics for initial data placement.

Authors 6

1. Jannis Klinkenberg corresponding Aachen Chair for Computer Science 12

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Chair for Computer Science 12, Seffenter Weg 23, Aachen, 52074, NRW, Germany

2. Anara Kozhokanova Aachen Chair for Computer Science 12

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Chair for Computer Science 12, Seffenter Weg 23, Aachen, 52074, NRW, Germany

3. Christian Terboven Aachen Chair for Computer Science 12

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Chair for Computer Science 12, Seffenter Weg 23, Aachen, 52074, NRW, Germany

4. Clément Foyer

   Université de Bordeaux · Institut national de recherche en sciences et technologies du numérique · Université de Reims Champagne-Ardenne · Laboratoire Bordelais de Recherche en Informatique · Laboratoire d'Informatique en Calcul Intensif et Image pour la Simulation

   Affiliation as printed

   Inria, Univ. Bordeaux, LaBRI, Talence, France

   Université de Reims Champagne-Ardenne, LICIIS, LRC DIGIT, Reims, 51097, France

5. Brice Goglin

   Université de Bordeaux · Institut national de recherche en sciences et technologies du numérique · Laboratoire Bordelais de Recherche en Informatique

   Affiliation as printed

   Inria, Univ. Bordeaux, LaBRI, Talence, France

6. Emmanuel Jeannot

   Université de Bordeaux · Institut national de recherche en sciences et technologies du numérique · Laboratoire Bordelais de Recherche en Informatique

   Affiliation as printed

   Inria, Univ. Bordeaux, LaBRI, Talence, France