Experimental quantum speed-up in reinforcement learning agents

Zenodo (CERN European Organization for Nuclear Research)

Abstract

The content of the text files can be used to reproduce the experimental plots presented in the manuscript. All the files contain sequences of numbers "0" and "1", which represent the non-reward and reward assigned to the agent after every epoch, respectively. In more detail, three different cases are shown: classical strategy, quantum strategy, and combined strategy. 1. Classical strategy The text files named "Classical_10agents_1000rounds.txt", "Classical_139agents_1000rounds.txt", and "Classical_16agents_1000rounds.txt" consist of 10, 139, and 16 consecutive sequences of 1000 numbers, respectively. Each of these sequences corresponds to one agent playing 1000 epochs. Therefore, there are in total 10+139+16=165 agents, each playing 1000 epochs. Merging these files and averaging the epoch outcomes 0 or 1 over all the 165 agents will make it possible to reproduce the behaviour of the average reward for the classical strategy. 2. Quantum strategy The text file named "Quantum_165agents_500rounds.txt" contains 165 arrays of length 500. Similarly to the previous case, the content of each array is progressively created every time the agent plays an epoch. 500 epochs are played. Also in this case, averaging the different outcomes over all the 165 agents will reproduce the behaviour of the average reward for the quantum strategy. 3. Combined strategy In this case, one file is acquired for the quantum strategy, and one for the classical strategy. These files are named "Combined_quantum_165agents.txt" and "Combined_classical_165agents.txt", respectively. Also in this case, 165 arrays (representing the 165 agents) are present in both files. However, the length of these arrays is not always the same. Taking as an example the first array of both the quantum and the classical files, there are 47 elements in the quantum case and 906 in the classical case. This means that the agent has played 47 quantum epochs before switching to a classical strategy. Therefore, in order to combine these two cases, one needs to merge the quantum array with the classical array, where only the even-indexed elements have been selected in the classical file. In this way, one obtains 47+906/2=500 epochs. The same procedure applies to the rest of the arrays. After 165 arrays of length 500 are obtained, the average over all the agents can be performed, and the plot for the combined strategy can thus be reproduced.

Authors 11

1. Valeria Saggio

   University of Vienna

   Affiliation as printed

   University of Vienna

2. Beate E. Asenbeck

   University of Vienna

   Affiliation as printed

   University of Vienna

3. Teodor Strömberg

   University of Vienna

   Affiliation as printed

   University of Vienna

4. Peter Schiansky

   University of Vienna

   Affiliation as printed

   University of Vienna

5. Vedran Dunjko Aachen

   Leiden University

   Affiliation as printed

   Leiden University

6. Nicolai Friis
   Affiliation as printed

   IQOQI Vienna

7. Nicholas C. Harris

   Massachusetts Institute of Technology

   Affiliation as printed

   Massachusetts Institute of Technology

8. Dirk Englund

   Massachusetts Institute of Technology

   Affiliation as printed

   Massachusetts Institute of Technology

9. Sabine Wölk

   Universität Innsbruck

   Affiliation as printed

   University of Innsbruck

10. Hans J. Briegel

    Universität Innsbruck

    Affiliation as printed

    University of Innsbruck

11. Philip John Walther

    University of Vienna

    Affiliation as printed

    University of Vienna