Parallel implementation and performance of super-resolution generative adversarial network turbulence models for large-eddy simulation

Computers & Fluids, vol. 288, pp. 106498

Abstract

Super-resolution (SR) generative adversarial networks (GANs) are promising for turbulence closure in large-eddy simulation (LES) due to their ability to accurately reconstruct high-resolution data from low-resolution fields. Current model training and inference strategies are not sufficiently mature for large-scale, distributed calculations due to the computational demands and often unstable training of SR-GANs, which limits the exploration of improved model structures, training strategies, and loss-function definitions. Integrating SR-GANs into LES solvers for inference-coupled simulations is also necessary to assess their a posteriori accuracy, stability, and cost. We investigate parallelization strategies for SR-GAN training and inference-coupled LES, focusing on computational performance and reconstruction accuracy. We examine distributed data-parallel training strategies for hybrid CPU–GPU node architectures and the associated influence of low-/high-resolution subbox size, global batch size, and discriminator accuracy. Accurate predictions require training subboxes that are sufficiently large relative to the Kolmogorov length scale. Care should be placed on the coupled effect of training batch size, learning rate, number of training subboxes, and discriminator’s learning capabilities. We introduce a data-parallel SR-GAN training and inference library for heterogeneous architectures that enables exchange between the LES solver and SR-GAN inference at runtime. We investigate the predictive accuracy and computational performance of this arrangement with particular focus on the overlap (halo) size required for accurate SR reconstruction. Similarly, a posteriori parallel scaling for efficient inference-coupled LES is constrained by the SR subdomain size, GPU utilization, and reconstruction accuracy. Based on these findings, we establish guidelines and best practices to optimize resource utilization and parallel acceleration of SR-GAN turbulence model training and inference-coupled LES calculations while maintaining predictive accuracy.

Authors 8

1. Ludovico Nista corresponding Aachen Institute for Combustion Technology

   RWTH Aachen University

   Affiliation as printed

   Institute for Combustion Technology, RWTH Aachen University, Aachen, 52056, Germany

2. Christoph Schumann

   University of Cambridge

   Affiliation as printed

   Department of Engineering, University of Cambridge, Cambridge, CB2 1PZ, United Kingdom

3. P. Petkov

   National Center for Supercomputing Applications · Sofia University "St. Kliment Ohridski"

   Affiliation as printed

   Faculty of Physics, Sofia University St. Kliment Ohridski, Sofia, 1164, Bulgaria

   National Centre for Supercomputing Applications, Sofia, 1113, Bulgaria

4. Temistocle Grenga

   University of Southampton

   Affiliation as printed

   Department of Aeronautics and Astronautics, Faculty of Engineering and Physical Sciences, University of Southampton, Southampton, SO17 1BJ, United Kingdom

5. Jonathan F. MacArt

   University of Notre Dame

   Affiliation as printed

   Aerospace and Mechanical Engineering, University of Notre Dame, Notre Dame, IN 46556, USA

6. Antonio Attili

   University of Edinburgh

   Affiliation as printed

   School of Engineering, Institute for Multiscale Thermofluids, University of Edinburgh, Edinburgh, EH93FD, United Kingdom

7. Stoyan Stefanov Markov

   National Center for Supercomputing Applications

   Affiliation as printed

   National Centre for Supercomputing Applications, Sofia, 1113, Bulgaria

8. Heinz G. Pitsch Aachen Institute for Combustion Technology

   RWTH Aachen University

   Affiliation as printed

   Institute for Combustion Technology, RWTH Aachen University, Aachen, 52056, Germany