D5.2: Specification of QRL algorithm for inventory management

Zenodo (CERN European Organization for Nuclear Research)

Abstract

The main objective of this deliverable is the theoretical development of quantum machine learning machinery for reinforcement learning, which is near-term-device friendly, yet sufficiently general to be applicable to the task of inventory management. This deliverable documents this achievement. With the advent of real-world quantum computing, the idea that parametrized quantum computations can be used as hypothesis families in a quantum-classical machine learning system is gaining increasing traction. Such hybrid quantum-classical systems, which use a classical loop over a parameterized quantum circuit, have already shown the potential to tackle real-world tasks in supervised and generative learning, and recent works have established their provable advantages in special artificial tasks. Yet, in the case of reinforcement learning, which is arguably most challenging and where learning boosts would be extremely valuable, no proposal has been successful in solving even standard benchmarking tasks, nor in showing a theoretical learning advantage over classical algorithms. In this work, we achieve both. We propose a hybrid quantum-classical reinforcement learning model using very few qubits, which we show can be effectively trained to solve several standard benchmarking environments. Moreover, we demonstrate, and formally prove, the ability of parametrized quantum circuits to solve certain learning tasks that are intractable to classical models, including current state-of-art deep neural networks, under the widely-believed classical hardness of the discrete logarithm problem. Our approach is general, and has shown applicable to both discrete and continuous state domains, with discrete actions, and is thus sufficiently general for application to the problem of inventory management. Furthermore, it is built around parametrized circuits, and is thus within the accepted paradigm for near-term-friendly proposals

Authors 2

1. Vedran Dunjko Aachen

   Leiden University

   Affiliation as printed

   Leiden University

2. Simon C. Marshall Aachen

   Leiden University

   Affiliation as printed

   Leiden University