Actor-Critic強化学習を組み込んだMPCによる自律ホイールローダの目標姿勢ナビゲーション
Autonomous Wheel Loader Navigation Using Goal-Conditioned Actor-Critic MPC
強化学習のActor-Criticで学習した価値関数をMPCのコストとして組み込み、ホイールローダを任意の目標姿勢へ効率的にナビゲーションする制御手法を提案し、実機でも検証した。
著者: Aleksi Mäki-Penttilä, Naeim Ebrahimi Toulkani, Reza Ghabcheloo
分類: cs.RO, cs.SY, eess.SY
原文アブストラクト
This paper proposes a novel control method for an autonomous wheel loader, enabling time-efficient navigation to an arbitrary goal pose. Unlike prior works which combine high-level trajectory planners with Model Predictive Control (MPC), we directly enhance the planning capabilities of MPC by incorporating a cost function derived from Actor-Critic Reinforcement Learning (RL). Specifically, we first train an RL agent to solve the pose reaching task in simulation, then transfer the learned planning knowledge to an MPC by incorporating the trained neural network critic as both the stage and terminal cost. We show through comprehensive simulations that the resulting MPC inherits the time-efficient behavior of the RL agent, generating trajectories that compare favorably against those found using trajectory optimization. We also deploy our method on a real-world wheel loader, where we demonstrate successful navigation in various scenarios.
