双方向プログレッシブニューラルネットワークとエピソードリターン進捗による創発的タスク順序付けとロボットスキル転移
Bidirectional Progressive Neural Networks with Episodic Return Progress for Emergent Task Sequencing and Robotic Skill Transfer
人間のスキル獲得と転移に着想を得て、エピソードリターン進捗に基づく内在的動機で自律的にタスクを切り替え、双方向にスキル転移できるマルチタスク強化学習フレームワークERP-BPNNを提案し、異なる形態のロボット間でのリーチングタスクで有効性を示した。
著者: Suzan Ece Ada, Hanne Say, Emre Ugur, Erhan Oztop
分類: cs.RO, cs.AI, cs.LG
原文アブストラクト
Human brain and behavior provide a rich venue that can inspire novel control and learning methods for robotics. In an attempt to exemplify such a development by inspiring how humans acquire knowledge and transfer skills among tasks, we introduce a novel multi-task reinforcement learning framework named Episodic Return Progress with Bidirectional Progressive Neural Networks (ERP-BPNN). The proposed ERP-BPNN model (1) learns in a human-like interleaved manner by (2) autonomous task switching based on a novel intrinsic motivation signal and, in contrast to existing methods, (3) allows bidirectional skill transfer among tasks. ERP-BPNN is a general architecture applicable to several multi-task learning settings; in this paper, we present the details of its neural architecture and show its ability to enable effective learning and skill transfer among morphologically different robots in a reaching task. The developed Bidirectional Progressive Neural Network (BPNN) architecture enables bidirectional skill transfer without requiring incremental training and seamlessly integrates with online task arbitration. The task arbitration mechanism developed is based on soft Episodic Return progress (ERP), a novel intrinsic motivation (IM) signal. To evaluate our method, we use quantifiable robotics metrics such as 'expected distance to goal' and 'path straightness' in addition to the usual reward-based measure of episodic return common in reinforcement learning. With simulation experiments, we show that ERP-BPNN achieves faster cumulative convergence and improves performance in all metrics considered among morphologically different robots compared to the baselines.
関連論文
- T3S: タスク固有特徴セレクタとスケジューラによるマルチタスク強化学習の改善マルチタスク強化学習
- 大規模マルチタスク世界モデルによる連続制御の学習マルチタスク強化学習
- 時間的に拡張された行動共有によるマルチタスク強化学習マルチタスク強化学習
- クロスタスク方策ガイダンスによる効率的なマルチタスク強化学習マルチタスク強化学習
- ロボティクス向け大規模並列マルチタスク強化学習のベンチマークマルチタスク強化学習
- 大きく、正則化し、カテゴリカルに:高容量価値関数は効率的なマルチタスク学習者マルチタスク強化学習
