Kernel Float: Unlocking Mixed-Precision GPU Programming

ACM Transactions on Mathematical Software, vol. 52, pp. 1–36

Abstract

Modern GPUs feature specialized hardware for low-precision floating-point arithmetic to accelerate compute-intensive workloads that do not require high numerical accuracy, such as those from artificial intelligence. However, despite the significant gains in computational throughput, memory bandwidth utilization, and energy efficiency, integrating low-precision formats into scientific applications remains difficult. We introduce Kernel Float , a header-only C++ library that simplifies the development of portable mixed-precision GPU kernels. Kernel Float provides a generic vector type, a unified interface for common mathematical operations, and fast approximations for low-precision transcendental functions that lack native hardware support. To demonstrate the potential of mixed-precision computing unlocked by our library, we integrated Kernel Float into nine GPU kernels from various domains. Our evaluation on Nvidia A100 and AMD MI250X GPUs shows performance improvements of up to \(12\times\) over double precision, while reducing source code length by up to 50% compared to handwritten kernels and having negligible runtime overhead. Our results further show that mixed-precision performance depends not only on choosing appropriate data types but also on tuning traditional optimization parameters (e.g., block size and vector width) and, when relevant, even domain-specific parameters.

Authors 2

1. Stijn Heldens

   Netherlands eScience Center

   Affiliation as printed

   Netherlands eScience Center, Amsterdam, The Netherlands

   Netherlands eScience Center, The Netherlands

2. Ben van Werkhoven Aachen

   Leiden University

   Affiliation as printed

   Leiden University, Leiden, The Netherlands

   Leiden University, The Netherlands