シミュレーションで学び、実車で走る
Learning from Simulation, Racing in Reality
簡単な車両モデルとモデルランダム化を用いたシミュレーションのみで強化学習ポリシーを訓練し、実車のミニチュアレースカーに転移させた。ポリシー出力の正則化と持ち上げアクション空間により滑らかかつアグレッシブな走行を実現し、実環境データで微調整することでMPCに匹敵するラップタイムと50%の制約違反削減を達成した。
著者: Eugenio Chisari, Alexander Liniger, Alisa Rupenyan, Luc Van Gool, John Lygeros
分類: cs.RO, cs.AI, cs.SY, eess.SY
原文アブストラクト
We present a reinforcement learning-based solution to autonomously race on a miniature race car platform. We show that a policy that is trained purely in simulation using a relatively simple vehicle model, including model randomization, can be successfully transferred to the real robotic setup. We achieve this by using novel policy output regularization approach and a lifted action space which enables smooth actions but still aggressive race car driving. We show that this regularized policy does outperform the Soft Actor Critic (SAC) baseline method, both in simulation and on the real car, but it is still outperformed by a Model Predictive Controller (MPC) state of the art method. The refinement of the policy with three hours of real-world interaction data allows the reinforcement learning policy to achieve lap times similar to the MPC controller while reducing track constraint violations by 50%.
