学習済みスキルとダイナミクスを用いた先読み探索によるマニピュレーション強化学習
Model Learning for Look-ahead Exploration in Continuous Control
多目標RLで学習したスキルとその粗いダイナミクスを先読み探索に用い、探索を誘導することで複雑なマニピュレーション方策を効率的に学習する手法を提案した論文。
著者: Arpit Agarwal, Katharina Muelling, Katerina Fragkiadaki
分類: cs.RO, cs.AI, cs.LG
原文アブストラクト
We propose an exploration method that incorporates look-ahead search over basic learnt skills and their dynamics, and use it for reinforcement learning (RL) of manipulation policies . Our skills are multi-goal policies learned in isolation in simpler environments using existing multigoal RL formulations, analogous to options or macroactions. Coarse skill dynamics, i.e., the state transition caused by a (complete) skill execution, are learnt and are unrolled forward during lookahead search. Policy search benefits from temporal abstraction during exploration, though itself operates over low-level primitive actions, and thus the resulting policies does not suffer from suboptimality and inflexibility caused by coarse skill chaining. We show that the proposed exploration strategy results in effective learning of complex manipulation policies faster than current state-of-the-art RL methods, and converges to better policies than methods that use options or parametrized skills as building blocks of the policy itself, as opposed to guiding exploration. We show that the proposed exploration strategy results in effective learning of complex manipulation policies faster than current state-of-the-art RL methods, and converges to better policies than methods that use options or parameterized skills as building blocks of the policy itself, as opposed to guiding exploration.
関連論文
- フロー反転アクションアンカーによるサンプル効率の高い強化学習方策改善強化学習/マニピュレーション
- フィードバック補正を備えたアクションチャンキングPPO強化学習/マニピュレーション
- 協調デルタ・3-RRSパラレルロボット挿入のための運動学認識型レインボー深層Q学習強化学習/マニピュレーション
- CONTHER: 後知恵経験再生とTransformerによる専門家デモ不要の人間らしい文脈的ロボット学習強化学習/マニピュレーション
- 多様性に基づく軌跡と目標選択を用いたHindsight Experience Replay強化学習/マニピュレーション
- 目標分布で条件付けする汎用強化学習ポリシー強化学習/マニピュレーション
