好奇心メタコントローラ：深層強化学習におけるモデルベース制御とモデルフリー制御の適応的切り替え
Curious Meta-Controller: Adaptive Alternation between Model-Based and Model-Free Control in Deep Reinforcement Learning
潜在空間でのダイナミクスモデルの学習進捗に基づく好奇心フィードバックを用いて、モデルベース制御とモデルフリー制御を適応的に切り替える手法を提案し、画像入力からのロボットのリーチング・把持タスクでサンプル効率と性能を改善した。
著者: Muhammad Burhan Hafez, Cornelius Weber, Matthias Kerzel, Stefan Wermter
分類: cs.LG, cs.AI, cs.RO, stat.ML
原文アブストラクト
Recent success in deep reinforcement learning for continuous control has been dominated by model-free approaches which, unlike model-based approaches, do not suffer from representational limitations in making assumptions about the world dynamics and model errors inevitable in complex domains. However, they require a lot of experiences compared to model-based approaches that are typically more sample-efficient. We propose to combine the benefits of the two approaches by presenting an integrated approach called Curious Meta-Controller. Our approach alternates adaptively between model-based and model-free control using a curiosity feedback based on the learning progress of a neural model of the dynamics in a learned latent space. We demonstrate that our approach can significantly improve the sample efficiency and achieve near-optimal performance on learning robotic reaching and grasping tasks from raw-pixel input in both dense and sparse reward settings.
