MSVIPER: 強化学習ベースのロボットナビゲーションのための改良された方策蒸留
MSVIPER: Improved Policy Distillation for Reinforcement-Learning-Based Robot Navigation
強化学習で学習したナビゲーション方策を決定木に蒸留し、動的障害物や不整地での凍結・振動を大幅に削減する手法を提案。
著者: Aaron M. Roth, Jing Liang, Ram Sriram, Elham Tabassi, Dinesh Manocha
分類: cs.RO, cs.AI, cs.HC, cs.LG
原文アブストラクト
We present Multiple Scenario Verifiable Reinforcement Learning via Policy Extraction (MSVIPER), a new method for policy distillation to decision trees for improved robot navigation. MSVIPER learns an "expert" policy using any Reinforcement Learning (RL) technique involving learning a state-action mapping and then uses imitation learning to learn a decision-tree policy from it. We demonstrate that MSVIPER results in efficient decision trees and can accurately mimic the behavior of the expert policy. Moreover, we present efficient policy distillation and tree-modification techniques that take advantage of the decision tree structure to allow improvements to a policy without retraining. We use our approach to improve the performance of RL-based robot navigation algorithms for indoor and outdoor scenes. We demonstrate the benefits in terms of reduced freezing and oscillation behaviors (by up to 95\% reduction) for mobile robots navigating among dynamic obstacles and reduced vibrations and oscillation (by up to 17\%) for outdoor robot navigation on complex, uneven terrains.
