内在的動機に基づくメタ制御と潜在空間経験想像によるロボット二重系運動学習の改善
Improving Robot Dual-System Motor Learning with Intrinsically Motivated Meta-Control and Latent-Space Experience Imagination
モデルベースとモデルフリーの判断をモデルの局所的信頼度推定でオンライン調停し、潜在空間での想像経験も学習に活用する二重系運動学習手法を提案し、視覚ベースの把持タスクで有効性を示した。
著者: Muhammad Burhan Hafez, Cornelius Weber, Matthias Kerzel, Stefan Wermter
分類: cs.LG, cs.AI, cs.RO, stat.ML
原文アブストラクト
Combining model-based and model-free learning systems has been shown to improve the sample efficiency of learning to perform complex robotic tasks. However, dual-system approaches fail to consider the reliability of the learned model when it is applied to make multiple-step predictions, resulting in a compounding of prediction errors and performance degradation. In this paper, we present a novel dual-system motor learning approach where a meta-controller arbitrates online between model-based and model-free decisions based on an estimate of the local reliability of the learned model. The reliability estimate is used in computing an intrinsic feedback signal, encouraging actions that lead to data that improves the model. Our approach also integrates arbitration with imagination where a learned latent-space model generates imagined experiences, based on its local reliability, to be used as additional training data. We evaluate our approach against baseline and state-of-the-art methods on learning vision-based robotic grasping in simulation and real world. The results show that our approach outperforms the compared methods and learns near-optimal grasping policies in dense- and sparse-reward environments.
