単一の観察デモンストレーションから学ぶ強化学習型逆動力学モデリング
RIDM: Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstration
デモンストレータの行動情報を必要とせず、単一の観察軌道から逆動力学モデルを強化学習で学習するRIDMを提案し、シミュレーションと実機UR5で有効性を示した。
著者: Brahma S. Pavse, Faraz Torabi, Josiah P. Hanna, Garrett Warnell, Peter Stone
分類: cs.LG, cs.RO, stat.ML
原文アブストラクト
Augmenting reinforcement learning with imitation learning is often hailed as a method by which to improve upon learning from scratch. However, most existing methods for integrating these two techniques are subject to several strong assumptions---chief among them that information about demonstrator actions is available. In this paper, we investigate the extent to which this assumption is necessary by introducing and evaluating reinforced inverse dynamics modeling (RIDM), a novel paradigm for combining imitation from observation (IfO) and reinforcement learning with no dependence on demonstrator action information. Moreover, RIDM requires only a single demonstration trajectory and is able to operate directly on raw (unaugmented) state features. We find experimentally that RIDM performs favorably compared to a baseline approach for several tasks in simulation as well as for tasks on a real UR5 robot arm. Experiment videos can be found at https://sites.google.com/view/ridm-reinforced-inverse-dynami.
