Hyperparameter Optimization and Neuroevolution with Binary Quadratic Meta-heuristics and Evolution Strategies

IEEE Congress on Information Science and Technology (CiSt), vol. 13, pp. 536–540

Abstract

This study extends previous work on quantum-enhanced selection operators [Von Dollen et al.(2022)], where selection operators of evolutionary algorithms for classical black-box optimization are framed as quadratic binary optimization (QUBO) models, applied to black-box objective functions. We test our algorithms using both elitist and non-elitist strategies. First, we benchmark all heuristics over black-box, multi-modal functions. We find that for 94.4% of test cases, our method shows performance improvements with respect to fitness. Then, we similarly benchmark over the problem of hyperparameter selection for machine learning models, also a black-box optimization problem, and compare it to random search. We find that our method achieves some improvement with an average validation accuracy of 98.492% and highest ranking of 4.529412 (out of 5) when averaged across all test cases. We also apply our method towards neuroevolution, where we evolve weight spaces for small feedforward neural networks for learning tasks. In these settings, our method shows improved average reward in the reinforcement learning case and increased velocity in learning the XOR function.

Authors 4

1. David Von Dollen Aachen

   Leiden University

   Affiliation as printed

   Leiden University,Leiden,The Netherlands

   Leiden University, Leiden, The Netherlands

2. Reuben Brasher

   Terra Quantum (Switzerland)

   Affiliation as printed

   Terra Quantum,Zurich,Switzerland

   Terra Quantum, Zurich, Switzerland

3. Florian Neukart

   Terra Quantum (Switzerland)

   Affiliation as printed

   Terra Quantum,Zurich,Switzerland

   Terra Quantum, Zurich, Switzerland

4. Thomas Bäck Aachen

   Leiden University

   Affiliation as printed

   Leiden University,Leiden,The Netherlands

   Leiden University, Leiden, The Netherlands