Bayesian experimental design in production engineering: a comprehensive performance and robustness study

Frontiers in Manufacturing Technology, vol. 5

Abstract

In production engineering, the identification of optimal process parameters is essential to advance product quality and overall equipment effectiveness. Optimizing and adapting process parameters through experimental design is relevant for different phases of the life cycle of a production process: (i) design and development of new processes, (ii) failure analysis and optimization, and (iii) adaptation and calibration in series production. Existing experimental design approaches tend to be inefficient because they comprise static, non-adaptive methodologies that separate experiment design from execution and analysis. Instead, Bayesian Optimization (BO) offers an adaptive and data-efficient methodology for experimental design termed Bayesian experimental design (BED). In BED, the selection of an experiment is re-evaluated in each iteration based on previous experiment results according to an acquisition function that aims to maximize the informational content of each experiment. However, the configuration of BO algorithms for specific optimization problems requires extensive knowledge of both BO and process characteristics. The mean and covariance functions of the surrogate model, the acquisition function, and initial data sampling must be individually configured and significantly influence overall optimization performance, preventing widespread adoption in production engineering practice. To guide the configuration of BO algorithms for optimizing production processes, in this paper, we perform an extensive benchmark study with a total of 15,360 experiments. We evaluate the performance of a variety of BO algorithm configurations (including kernels, acquisition functions, and initial sampling sizes) on a total of eight optimization problems with a noiseless and a noisy variant each. The performance and robustness analysis reveals significant performance differences between individual BO algorithm configurations. The results of our benchmarking serve as empirical references based on which we derive actionable guidelines for the application of BED in production engineering.

Authors 6

1. Lars Leyendecker corresponding Aachen Production Quality Fraunhofer Institute for Production Technology IPT

   Fraunhofer Institute for Production Technology IPT

   Affiliation as printed

   Production Quality, Fraunhofer Institute for Production Technology IPT, Aachen, Germany

2. Ana Maria Gonzalez Degetau Aachen Production Quality Fraunhofer Institute for Production Technology IPT

   Fraunhofer Institute for Production Technology IPT

   Affiliation as printed

   Production Quality, Fraunhofer Institute for Production Technology IPT, Aachen, Germany

3. Katharina Bata

   Karlsruhe Institute of Technology

   Affiliation as printed

   Karlsruhe Institute of Technology KIT, Scientific Computing Center (SCC), Karlsruhe, Germany

4. Jessica Emonts Aachen Department of Mechanical Engineering

   FH Aachen

   Affiliation as printed

   Department of Mechanical Engineering, University of Applied Sciences Aachen, Aachen, Germany

5. Angela Schmitz

   TH Köln - University of Applied Sciences

   Affiliation as printed

   Institute of Product Development and Engineering Design, Faculty of Process Engineering, Energy and Mechanical Systems, TH Köln—University of Applied Sciences, Köln, Germany

6. Robert Heinrich Schmitt Aachen Laboratory for Machine Tools and Production Engineering (WZL) Production Quality Fraunhofer Institute for Production Technology IPT

   Fraunhofer Institute for Production Technology IPT · RWTH Aachen University

   Affiliation as printed

   Laboratory for Machine Tools and Production Engineering (WZL) of RWTH Aachen University, Aachen, Germany

   Production Quality, Fraunhofer Institute for Production Technology IPT, Aachen, Germany