Out-of-sample estimation for a branch-and-bound algorithm with growing datasets

Journal of Global Optimization, vol. 92, pp. 615–642

Abstract

Abstract In [Sass et al., Eur. J. Oper. Res., 316 (1): 36 – 45, 2024], we proposed a branch-and-bound (B&B) algorithm with growing datasets for the deterministic global optimization of parameter estimation problems based on large datasets. Therein, we start the B&B algorithm with a reduced dataset and augment it until reaching the full dataset upon convergence. However, convergence may be slowed down by a gap between the lower bounds of the reduced and the original problem, in particular for noisy measurement data. Thus, we propose the use of out-of-sample estimation for improving the lower bounds calculated with reduced datasets. Based on this, we extend the deterministic approach and propose two heuristic approaches. The computational performance of all approaches is compared with the standard B&B algorithm as a benchmark based on real-world estimation problems from process systems engineering, biochemistry, and machine learning covering datasets with and without measurement noise. Our results indicate that the heuristic approaches can improve the final lower bounds on the optimal objective value without cutting off the global solution. Aside from this, we prove that resampling can decrease the variance of the lower bounds calculated based on random initial datasets. In our case study, resampling hardly affects the performance of the approaches which indicates that the B&B algorithm with growing datasets does not suffer from large variances.

Authors 4

1. Susanne Saß Aachen Process Systems Engineering (AVT.SVT)

   RWTH Aachen University

   Affiliation as printed

   Process Systems Engineering (AVT.SVT), RWTH Aachen University, 52074, Aachen, Germany

2. Alexander Mitsos Aachen Process Systems Engineering (AVT.SVT)

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · RWTH Aachen University

   Affiliation as printed

   Institute of Climate and Energy Systems: Energy Systems Engineering (ICE-1), Forschungszentrum Jülich GmbH, 52425, Jülich, Germany

   JARA-CSD, 52056, Aachen, Germany

   Process Systems Engineering (AVT.SVT), RWTH Aachen University, 52074, Aachen, Germany

3. Nikolay Ivanov Nikolov Aachen Institute of Statistics

   Bulgarian Academy of Sciences · Institute of Mathematics and Informatics · RWTH Aachen University

   Affiliation as printed

   Institute of Mathematics and Informatics, Bulgarian Academy of Sciences, 1113, Sofia, Bulgaria

   Institute of Statistics, RWTH Aachen University, 52056, Aachen, Germany

4. Angelos Tsoukalas corresponding

   Erasmus University Rotterdam

   Affiliation as printed

   Department of Technology and Operations Management, RSM Erasmus University Rotterdam, 3062 PA, Rotterdam, Netherlands