移動障害物下におけるロボット軌道の安全強化学習
Safe Reinforcement Learning of Robot Trajectories in the Presence of Moving Obstacles
移動障害物を避けるための退避ポリシーを強化学習で事前学習し、他タスクの学習時に衝突リスクを推定して安全な代替行動を選択する手法を提案。実機実験でリアルタイムな安全軌道生成を実証した。
著者: Jonas Kiemel, Ludovic Righetti, Torsten Kröger, Tamim Asfour
分類: cs.RO
原文アブストラクト
In this paper, we present an approach for learning collision-free robot trajectories in the presence of moving obstacles. As a first step, we train a backup policy to generate evasive movements from arbitrary initial robot states using model-free reinforcement learning. When learning policies for other tasks, the backup policy can be used to estimate the potential risk of a collision and to offer an alternative action if the estimated risk is considered too high. No matter which action is selected, our action space ensures that the kinematic limits of the robot joints are not violated. We analyze and evaluate two different methods for estimating the risk of a collision. A physics simulation performed in the background is computationally expensive but provides the best results in deterministic environments. If a data-based risk estimator is used instead, the computational effort is significantly reduced, but an additional source of error is introduced. For evaluation, we successfully learn a reaching task and a basketball task while keeping the risk of collisions low. The results demonstrate the effectiveness of our approach for deterministic and stochastic environments, including a human-robot scenario and a ball environment, where no state can be considered permanently safe. By conducting experiments with a real robot, we show that our approach can generate safe trajectories in real time.
