Balance timing heuristics for dynamic load balancing

RWTH Publications (RWTH Aachen)

Abstract

Modern high-performance computing (HPC) applications typically use a hybrid Message Passing Interface (MPI) and OpenMP approach to achieve parallel execution. However, parallelizing applications often presents multiple challenges. One common challenge is load balancing. Load balancing is necessary because an imbalanced load results in an increased runtime and wasted compute resources. Nevertheless, achieving effective load balancing can be challenging and time-consuming. To facilitate this, application-independent approaches are available. However, these approaches often still require application modification. To reduce this effort an automatic load balancing approach was developed based on the Dynamic CoreBinding (DCB) library and the On-the-fly critical-path tool (OTF-CPT). This approach mitigates load imbalances by reassigning cores to processes on the same node. Thus, it does not require application modification. This thesis focuses on improving the balancing performance of the tool by introducing additional heuristics. These heuristics aim to reduce the tool’s overhead and enhance its balancing performance. Additionally, I extend the tool to automatically reduce energy consumption rather than runtime by reducing load imbalances. Furthermore, I evaluate the tool’s balancing performance using self-written test cases, as well as two proxy and two HPC applications. The evaluation proves the tool’s ability to reduce both runtime and energy consumption; however, its effectiveness depends on the application structure, as well as the number of processes and threads used per node.

Authors 1

1. Cedric Halfen corresponding Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen