深層逆強化学習による脚式ロボットの地形走破性モデリング
Energy-based Legged Robots Terrain Traversability Modeling via Deep Inverse Reinforcement Learning
脚式ロボットの地形走破性を、外界センサと固有受容センサの両方を統合した深層逆強化学習でモデル化し、エネルギー消費に基づく報酬設計を実現した。
著者: Lu Gan, Jessy W. Grizzle, Ryan M. Eustice, Maani Ghaffari
分類: cs.RO
原文アブストラクト
This work reports on developing a deep inverse reinforcement learning method for legged robots terrain traversability modeling that incorporates both exteroceptive and proprioceptive sensory data. Existing works use robot-agnostic exteroceptive environmental features or handcrafted kinematic features; instead, we propose to also learn robot-specific inertial features from proprioceptive sensory data for reward approximation in a single deep neural network. Incorporating the inertial features can improve the model fidelity and provide a reward that depends on the robot's state during deployment. We train the reward network using the Maximum Entropy Deep Inverse Reinforcement Learning (MEDIRL) algorithm and propose simultaneously minimizing a trajectory ranking loss to deal with the suboptimality of legged robot demonstrations. The demonstrated trajectories are ranked by locomotion energy consumption, in order to learn an energy-aware reward function and a more energy-efficient policy than demonstration. We evaluate our method using a dataset collected by an MIT Mini-Cheetah robot and a Mini-Cheetah simulator. The code is publicly available at https://github.com/ganlumomo/minicheetah-traversability-irl.
