Distributed hybrid quantum-classical performance prediction for hyperparameter optimization

Quantum Machine Intelligence, vol. 6

Abstract

Abstract Hyperparameter optimization (HPO) of neural networks is a computationally expensive procedure, which requires a large number of different model configurations to be trained. To reduce such costs, this work presents a distributed, hybrid workflow, that runs the training of the neural networks on multiple graphics processing units (GPUs) on a classical supercomputer, while predicting the configurations’ performance with quantum-trained support vector regression (QT-SVR) on a quantum annealer (QA). The workflow is shown to run on up to 50 GPUs and a QA at the same time, completely automating the communication between the classical and the quantum systems. The approach is evaluated extensively on several benchmarking datasets from the computer vision (CV), high-energy physics (HEP), and natural language processing (NLP) domains. Empirical results show that resource costs for performing HPO can be reduced by up to 9% when using the hybrid workflow with performance prediction, compared to using a plain HPO algorithm without performance prediction. Additionally, the workflow obtains similar and in some cases even better accuracy of the final hyperparameter configuration, when combining multiple heuristically obtained predictions from the QA, compared to using just a single classically obtained prediction. The results highlight the potential of hybrid quantum-classical machine learning algorithms. The workflow code is made available open-source to foster adoption in the community.

Authors 12

1. Eric Wulff corresponding

   European Organization for Nuclear Research

   Affiliation as printed

   CERN, Meyrin, Switzerland

2. Juan Pablo García Amboage
3. Marcel Aach

   University of Iceland · Forschungszentrum Jülich · Jülich Supercomputing Centre

   Affiliation as printed

   Jülich Supercomputing Centre, Forschungszentrum Jülich, Jülich, Germany

   School of Engineering and Natural Sciences, University of Iceland, Reykjavík, Iceland

4. Þorsteinn Elí Gíslason

   University of Iceland

   Affiliation as printed

   School of Engineering and Natural Sciences, University of Iceland, Reykjavík, Iceland

5. Thorsteinn Kristinn Ingolfsson

   University of Iceland

   Affiliation as printed

   School of Engineering and Natural Sciences, University of Iceland, Reykjavík, Iceland

6. Tomas Kristinn Ingolfsson

   University of Iceland

   Affiliation as printed

   School of Engineering and Natural Sciences, University of Iceland, Reykjavík, Iceland

7. Edoardo Pasetto Aachen

   Forschungszentrum Jülich · Jülich Supercomputing Centre · RWTH Aachen University

   Affiliation as printed

   Jülich Supercomputing Centre, Forschungszentrum Jülich, Jülich, Germany

   RWTH Aachen University, Aachen, Germany

8. Amer Delilbasic

   University of Iceland · Forschungszentrum Jülich · Jülich Supercomputing Centre

   Affiliation as printed

   Jülich Supercomputing Centre, Forschungszentrum Jülich, Jülich, Germany

   School of Engineering and Natural Sciences, University of Iceland, Reykjavík, Iceland

9. Morris Riedel

   University of Iceland · Forschungszentrum Jülich · Jülich Supercomputing Centre

   Affiliation as printed

   Jülich Supercomputing Centre, Forschungszentrum Jülich, Jülich, Germany

   School of Engineering and Natural Sciences, University of Iceland, Reykjavík, Iceland

10. Rakesh Sarma

    Forschungszentrum Jülich · Jülich Supercomputing Centre

    Affiliation as printed

    Jülich Supercomputing Centre, Forschungszentrum Jülich, Jülich, Germany

11. Maria Girone

    European Organization for Nuclear Research

    Affiliation as printed

    CERN, Meyrin, Switzerland

12. Andreas Lintermann

    Forschungszentrum Jülich · Jülich Supercomputing Centre

    Affiliation as printed

    Jülich Supercomputing Centre, Forschungszentrum Jülich, Jülich, Germany