Aligning Storage Benchmark Metrics with Application-Level Performance

Abstract

The current state of practice in HPC is that performance metrics reported by storage benchmarks are disconnected from those obtained through application-level performance analysis tools, making it difficult for users to determine whether performance tuning efforts are effective or whether observed performance indicates underutilization of the system. In this work, we propose an approach to align IO500 storage benchmark results with application-level performance by deconstructing benchmark components and recalculating their metrics. The results show that certain universal metrics, such as bandwidth, can be meaningfully aligned with application performance, enabling more consistent and interpretable evaluation. Our findings also identify metrics that remain missing or cannot be reconciled, highlighting the need for standardization to align metrics produced by benchmarks and performance analysis tools.

Authors 5

1. Radita Liem

   Johannes Gutenberg University Mainz

   Affiliation as printed

   Johannes Gutenberg University Mainz, Mainz, Rheinland-Pfalz, Germany

2. Khoa Nguyen Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Aachen, Germany

3. Julian Martin Kunkel

   University of Göttingen · Gesellschaft für wissenschaftliche Datenverarbeitung mbH Göttingen

   Affiliation as printed

   GWDG / University of Goettingen, Goettingen, Germany

4. Jay Lofstead

   Sandia National Laboratories

   Affiliation as printed

   Sandia National Laboratory, Albuquerque, USA

5. Sarah Neuwirth

   Johannes Gutenberg University Mainz

   Affiliation as printed

   Johannes Gutenberg University Mainz, Mainz, Germany