非定常な相互依存タスクの自律的オープンエンド学習
Autonomous Open-Ended Learning of Tasks with Non-Stationary Interdependencies
目標間の相互依存関係が変化する環境で、内的動機に基づきサブタスク選択をMDPとして扱う階層的アーキテクチャH-GRAILを提案し、効率的なスキル獲得を実現した。
著者: Alejandro Romero, Gianluca Baldassarre, Richard J. Duro, Vieri Giuliano Santucci
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
Autonomous open-ended learning is a relevant approach in machine learning and robotics, allowing the design of artificial agents able to acquire goals and motor skills without the necessity of user assigned tasks. A crucial issue for this approach is to develop strategies to ensure that agents can maximise their competence on as many tasks as possible in the shortest possible time. Intrinsic motivations have proven to generate a task-agnostic signal to properly allocate the training time amongst goals. While the majority of works in the field of intrinsically motivated open-ended learning focus on scenarios where goals are independent from each other, only few of them studied the autonomous acquisition of interdependent tasks, and even fewer tackled scenarios where goals involve non-stationary interdependencies. Building on previous works, we tackle these crucial issues at the level of decision making (i.e., building strategies to properly select between goals), and we propose a hierarchical architecture that treating sub-tasks selection as a Markov Decision Process is able to properly learn interdependent skills on the basis of intrinsically generated motivations. In particular, we first deepen the analysis of a previous system, showing the importance of incorporating information about the relationships between tasks at a higher level of the architecture (that of goal selection). Then we introduce H-GRAIL, a new system that extends the previous one by adding a new learning layer to store the autonomously acquired sequences of tasks to be able to modify them in case the interdependencies are non-stationary. All systems are tested in a real robotic scenario, with a Baxter robot performing multiple interdependent reaching tasks.
関連論文
- ロボットのオープンエンド学習課題に向けた動機付けアーキテクチャオープンエンド学習
- ユーザーの目的に焦点を当てたロボットのオープンエンド強化学習オープンエンド学習
- オープンエンド学習ロボットにおける自律性と整合性の問題:目的フレームワークの形式化オープンエンド学習
- REAL-X:真にエンドツーエンドな感覚運動自律学習を実現するロボットのオープンエンド学習アーキテクチャオープンエンド学習
- 相互依存タスクの自律的オープンエンド学習オープンエンド学習
