Benchmarking of hyperparameter optimization techniques for machine learning applications in production

Advances in Industrial and Manufacturing Engineering, vol. 5, pp. 100099

Abstract

Machine learning (ML) has become a key technology to leverage the potential of large data amounts that are generated in the context of digitized and connected production processes. In projects for developing ML solutions for production applications, the selection of hyperparameter optimization (HPO) techniques is a key task that significantly impacts the performance of the resulting ML solution. However, selecting the best suitable HPO technique for an ML use case is challenging, since HPO techniques have individual strengths and weaknesses and ML use cases in production are highly individual in terms of their application areas, objectives, and resources. This makes the selection of HPO techniques in production a very complex task that requires decision support. Thus, we present a structured approach for benchmarking HPO techniques and for integrating the empirical data generated within benchmarking experiments into decision support systems. Based on the data generated within a large-scale benchmarking study, the validation results prove that the usage of benchmarking data improves decision-making in HPO technique selection and thus helps to exploit the full potential of ML solutions in production applications.

Authors 3

1. Maximilian Motz corresponding Aachen Fraunhofer Institute for Production Technology IPT

   Fraunhofer Institute for Production Technology IPT

   Affiliation as printed

   Fraunhofer Institute for Production Technology IPT, Steinbachstraße 17, Aachen, 52074, Germany

2. Jonathan Krauß

   Fraunhofer Research Institution for Battery Cell Production

   Affiliation as printed

   Fraunhofer Research Institution for Battery Cell Production FFB, Mendelstraße 11, Münster, 48149, Germany

3. Robert Heinrich Schmitt Aachen Fraunhofer Institute for Production Technology IPT Laboratory for Machine Tools WZL

   RWTH Aachen University · Fraunhofer Institute for Production Technology IPT

   Affiliation as printed

   Fraunhofer Institute for Production Technology IPT, Steinbachstraße 17, Aachen, 52074, Germany

   Laboratory for Machine Tools WZL RWTH Aachen University, Steinbachstraße 19, Aachen, 52074, Germany