Using drones as reference sensors for neural-networks-based modeling of automotive perception errors

IEEE Intelligent Vehicles Symposium, pp. 708–715

Abstract

Modeling perception errors of automated vehicles requires reference data, but common reference measurement methods either cannot capture uninstructed road users or suffer from vehicle-vehicle-occlusions. Therefore, we propose a method based on a camera-equipped drone hovering over the field of view of the perception system that is to be modeled. From recordings of this advantageous perspective, computer vision algorithms extract object tracks suited as reference. As a proof of concept of our approach, we create and analyze a phenomenological error model of a lidar-based sensor system. From eight hours of simultaneous traffic recordings at an intersection, we extract synchronized state vectors of associated true-positive vehicle tracks. We model the deviations of the full lidar state vectors from the reference as multivariate Gaussians. The dependency of their covariance matrices and mean vectors on the reference state vector is modeled by a fully-connected neural network. By customizing the network training procedure and losses, we are able to achieve consistent results even in sparsely populated areas of the state space. Finally, we show that time dependencies of errors can be considered separately during sampling by an autoregressive model.

Authors 6

1. Robert Krajewski Aachen Research area Vehicle Intelligence & Automated Driving Institute for Automotive Engineering

   RWTH Aachen University

   Affiliation as printed

   Research area Vehicle Intelligence & Automated Driving, Institute for Automotive Engineering, RWTH Aachen University, Aachen, Germany

2. Michael Hoss Aachen Vehicle Intelligence & Automated Driving Institute for Automotive Engineering

   RWTH Aachen University

   Affiliation as printed

   Vehicle Intelligence & Automated Driving, Institute for Automotive Engineering, RWTH Aachen University, Aachen, Germany

3. Adrian Meister

   ETH Zurich · Institute for Transport Planning and Systems

   Affiliation as printed

   Institute for Transport Planning and Systems, ETH Zurich, Zurich, Switzerland

4. Fabian Thomsen Aachen Vehicle Intelligence & Automated Driving Institute for Automotive Engineering

   RWTH Aachen University

   Affiliation as printed

   Vehicle Intelligence & Automated Driving, Institute for Automotive Engineering, RWTH Aachen University, Aachen, Germany

5. Julian Bock Aachen fka GmbH

   Forschungsgesellschaft Kraftfahrwesen Aachen (Germany)

   Affiliation as printed

   Fka GmbH, Aachen, Germany

6. Lutz Eckstein Aachen Vehicle Intelligence & Automated Driving Institute for Automotive Engineering

   RWTH Aachen University

   Affiliation as printed

   Vehicle Intelligence & Automated Driving, Institute for Automotive Engineering, RWTH Aachen University, Aachen, Germany