例示ベースのリセットで強化学習を自動化する
Automating Reinforcement Learning with Example-based Resets
リセット操作を自己教師あり学習で行う追加エージェントを導入し、実世界ロボットの強化学習における手動リセットを削減しつつ前向き方策の学習を進める手法を提案した。
著者: Jigang Kim, J. hyeon Park, Daesol Cho, H. Jin Kim
分類: cs.LG, cs.RO
原文アブストラクト
Deep reinforcement learning has enabled robots to learn motor skills from environmental interactions with minimal to no prior knowledge. However, existing reinforcement learning algorithms assume an episodic setting, in which the agent resets to a fixed initial state distribution at the end of each episode, to successfully train the agents from repeated trials. Such reset mechanism, while trivial for simulated tasks, can be challenging to provide for real-world robotics tasks. Resets in robotic systems often require extensive human supervision and task-specific workarounds, which contradicts the goal of autonomous robot learning. In this paper, we propose an extension to conventional reinforcement learning towards greater autonomy by introducing an additional agent that learns to reset in a self-supervised manner. The reset agent preemptively triggers a reset to prevent manual resets and implicitly imposes a curriculum for the forward agent. We apply our method to learn from scratch on a suite of simulated and real-world continuous control tasks and demonstrate that the reset agent successfully learns to reduce manual resets whilst also allowing the forward policy to improve gradually over time.
