First Experiences in Performance Benchmarking with the New SPEChpc 2021 Suites

IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid), pp. 675–684

Abstract

Modern High Performance Computing (HPC) sys-tems are built with innovative system architectures and novel programming models to further push the speed limit of computing. The increased complexity poses challenges for performance portability and performance evaluation. The Standard Perfor-mance Evaluation Corporation (SPEC) has a long history of producing industry-standard benchmarks for modern computer systems. SPEC's newly released SPEChpc 2021 benchmark suites, developed by the High Performance Group, are a bold attempt to provide a fair and objective benchmarking tool designed for state-of-the-art HPC systems. With the support of multiple host and accelerator programming models, the suites are portable across both homogeneous and heterogeneous architectures. Different workloads are developed to fit system sizes ranging from a few compute nodes to a few hundred compute nodes. In this work we present our first experiences in performance benchmarking the new SPEChpc2021 suites and evaluate their portability and basic performance characteristics on various popular and emerging HPC architectures, including x86 CPU, NVIDIA GPU, and AMD GPU. This study provides a first-hand experience of executing the SPEChpc 2021 suites at scale on production HPC systems, discusses real-world use cases, and serves as an initial guideline for using the benchmark suites.

Authors 10

1. Holger Brunst

   Technische Universität Dresden

   Affiliation as printed

   Technische Universität Dresden,Dresden,Germany

2. Sunita Chandrasekaran

   University of Delaware

   Affiliation as printed

   University of Delaware,Newark,DE,USA

   University of Delaware, Newark, DE, USA

3. Florina M. Ciorba

   University of Basel

   Affiliation as printed

   University of Basel,Basel,Switzerland

   University of Basel, Basel, Switzerland

4. Nick Hagerty

   Oak Ridge National Laboratory

   Affiliation as printed

   Oak Ridge National Laboratory,Oak Ridge,TN,USA

   Oak Ridge National Laboratory, Oak Ridge, TN, USA

5. Robert Henschel

   Indiana University Bloomington · Indiana University

   Affiliation as printed

   Indiana University,Bloomington,IN,USA

   Indiana University, Bloomington, IN, USA

6. Guido Juckeland

   Helmholtz-Zentrum Dresden-Rossendorf

   Affiliation as printed

   Helmholtz-Zentrum Dresden-Rossendorf,Dresden,Germany

   Helmholtz-Zentrum Dresden-Rossendorf, Dresden, Germany

7. Junjie Li

   Texas Advanced Computing Center · The University of Texas at Austin

   Affiliation as printed

   Texas Advanced Computing Center, The University of Texas at Austin,Austin,TX,USA

   Texas Advanced Computing Center, The University of Texas at Austin, Austin, TX, USA

8. Veronica Melesse Vergara

   Oak Ridge National Laboratory

   Affiliation as printed

   Oak Ridge National Laboratory,Oak Ridge,TN,USA

   Oak Ridge National Laboratory, Oak Ridge, TN, USA

9. Sandra Wienke Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Aachen,Germany

   RWTH Aachen University, Aachen, Germany

10. Miguel Zavala

    University of Delaware

    Affiliation as printed

    University of Delaware,Newark,DE,USA

    University of Delaware, Newark, DE, USA