Autonomous robotic nanofabrication with reinforcement learning

RWTH Publications (RWTH Aachen)

Abstract

The ability to handle single molecules as effectively as macroscopic building blocks would enable the construction of complex supramolecular structures inaccessible to self-assembly.The fundamental challenges obstructing this goal are the uncontrolled variability and poor observability of atomic-scale conformations.Here, we present a strategy to work around both obstacles and demonstrate autonomous robotic nanofabrication by manipulating single molecules.Our approach uses reinforcement learning (RL), which finds solution strategies even in the face of large uncertainty and sparse feedback.We demonstrate the potential of our RL approach by removing molecules autonomously with a scanning probe microscope from a supramolecular structure.Our RL agent reaches an excellent performance, enabling us to automate a task that previously had to be performed by a human.We anticipate that our work opens the way toward autonomous agents for the robotic construction of functional supramolecular structures with speed, precision, and perseverance beyond our current capabilities.

Authors 5

1. Philipp Leinen corresponding Aachen Experimentalphysik IV A

   Forschungszentrum Jülich · Jülich Aachen Research Alliance · RWTH Aachen University

   Affiliation as printed

   Experimentalphysik IV A, RWTH Aachen University, Otto-Blumenthal-Straße, 52074 Aachen, Germany

   Jülich Aachen Research Alliance (JARA)–Fundamentals of Future Information Technology, 52425 Jülich, Germany

   Peter Grünberg Institut (PGI-3), Forschungszentrum Jülich, 52425 Jülich, Germany

2. Malte Esders corresponding

   Technische Universität Berlin

   Affiliation as printed

   Machine Learning Group , Technische Universität Berlin , 10587 Berlin , Germany

   Machine Learning Group, Technische Universität Berlin, 10587 Berlin, Germany

3. Kristof T. Schütt

   Technische Universität Berlin

   Affiliation as printed

   Machine Learning Group , Technische Universität Berlin , 10587 Berlin , Germany

   Machine Learning Group, Technische Universität Berlin, 10587 Berlin, Germany

4. Christian Wagner corresponding

   Forschungszentrum Jülich · Jülich Aachen Research Alliance

   Affiliation as printed

   Jülich Aachen Research Alliance (JARA)-Fundamentals of Future Information Technology , 52425 Jülich , Germany

   Peter Grünberg Institut (PGI-3) , Forschungszentrum Jülich , 52425 Jülich , Germany

   Jülich Aachen Research Alliance (JARA)–Fundamentals of Future Information Technology, 52425 Jülich, Germany

   Peter Grünberg Institut (PGI-3), Forschungszentrum Jülich, 52425 Jülich, Germany

5. Klaus‐Robert Müller corresponding

   Korea University · Max Planck Institute for Informatics · Technische Universität Berlin

   Affiliation as printed

   Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea

   Machine Learning Group, Technische Universität Berlin, 10587 Berlin, Germany

   Max Planck Institute for Informatics, 66123 Saarbrücken, Germany