スキル空間計画によるリセット不要の生涯学習
Reset-Free Lifelong Learning with Skill-Space Planning
非エピソード的で非定常な環境でも破滅的失敗を避けつつ長期計画できるよう、高次スキルを教師なしで獲得し、学習したダイナミクスモデルでスキル空間上を計画するLiSPを提案した論文。
著者: Kevin Lu, Aditya Grover, Pieter Abbeel, Igor Mordatch
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
The objective of lifelong reinforcement learning (RL) is to optimize agents which can continuously adapt and interact in changing environments. However, current RL approaches fail drastically when environments are non-stationary and interactions are non-episodic. We propose Lifelong Skill Planning (LiSP), an algorithmic framework for non-episodic lifelong RL based on planning in an abstract space of higher-order skills. We learn the skills in an unsupervised manner using intrinsic rewards and plan over the learned skills using a learned dynamics model. Moreover, our framework permits skill discovery even from offline data, thereby reducing the need for excessive real-world interactions. We demonstrate empirically that LiSP successfully enables long-horizon planning and learns agents that can avoid catastrophic failures even in challenging non-stationary and non-episodic environments derived from gridworld and MuJoCo benchmarks.
