Zero-shot MARL benchmark in the Cyber-Physical Mobility Lab

at - Automatisierungstechnik, vol. 74, pp. 376–385

Abstract

Abstract We present a reproducible benchmark for evaluating sim-to-real transfer of Multi-Agent Reinforcement Learning (MARL) policies for Connected and Automated Vehicles (CAVs). The platform, based on the Cyber-Physical Mobility Lab (CPM Lab) (M. Kloock et al ., “Cyber-physical mobility lab: An open-source platform for networked and autonomous vehicles,” in European Control Conference (ECC) , 2021, pp. 1937–1944), integrates simulation, a high-fidelity digital twin, and a physical testbed, enabling structured zero-shot evaluation of MARL motion-planning policies. We demonstrate its use by deploying a SigmaRL-trained policy (J. Xu, P. Hu, and B. Alrifaee, “SigmaRL: A sample-efficient and generalizable multi-agent reinforcement learning framework for motion planning,” in IEEE International Conference on Intelligent Transportation Systems (ITSC) , 2024, pp. 768–775) across all three domains, revealing two complementary sources of performance degradation: architectural differences between simulation and hardware control stacks, and the sim-to-real gap induced by increasing environmental realism. The open-source setup enables systematic analysis of sim-to-real challenges in MARL under realistic, reproducible conditions.

Authors 5

1. Julius Beerwerth

   Universität der Bundeswehr München

   Affiliation as printed

   Department of Aerospace Engineering , 26576 University of the Bundeswehr Munich , 85579 Neubiberg , Germany

2. Jianye Xu Aachen Department of Computer Science

   RWTH Aachen University

   Affiliation as printed

   Department of Computer Science , RWTH Aachen University , 52074 Aachen , Germany

3. Simon Schäfer Aachen Department of Computer Science

   RWTH Aachen University

   Affiliation as printed

   Department of Computer Science , RWTH Aachen University , 52074 Aachen , Germany

4. Fynn Belderink Aachen Department of Computer Science

   RWTH Aachen University

   Affiliation as printed

   Department of Computer Science , RWTH Aachen University , 52074 Aachen , Germany

5. Bassam Alrifaee

   Universität der Bundeswehr München

   Affiliation as printed

   Department of Aerospace Engineering , 26576 University of the Bundeswehr Munich , 85579 Neubiberg , Germany