後継特徴量の並行合成による連続制御のマルチタスク強化学習
Multi-Task Reinforcement Learning in Continuous Control with Successor Feature-Based Concurrent Composition
後継特徴量の合成性を利用し、新たな方策学習なしにプリミティブから方策分布を合成するマルチタスク強化学習手法を提案し、IsaacGym上のベンチマークで単一タスク性能と未見タスクへの転移を実証した。
著者: Yu Tang Liu, Aamir Ahmad
分類: cs.RO
原文アブストラクト
Deep reinforcement learning (DRL) frameworks are increasingly used to solve high-dimensional continuous control tasks in robotics. However, due to the lack of sample efficiency, applying DRL for online learning is still practically infeasible in the robotics domain. One reason is that DRL agents do not leverage the solution of previous tasks for new tasks. Recent work on multi-task DRL agents based on successor features (SFs) has proven to be quite promising in increasing sample efficiency. In this work, we present a new approach that unifies two prior multi-task RL frameworks, SF-GPI and value composition, and adapts them to the continuous control domain. We exploit compositional properties of successor features to compose a policy distribution from a set of primitives without training any new policy. Lastly, to demonstrate the multi-tasking mechanism, we present our proof-of-concept benchmark environments, Pointmass and Pointer, based on IsaacGym, which facilitates large-scale parallelization to accelerate the experiments. Our experimental results show that our multi-task agent has single-task performance on par with soft actor-critic (SAC), and the agent can successfully transfer to new unseen tasks. We provide our code as open-source at "https://github.com/robot-perception-group/concurrent_composition" for the benefit of the community.
関連論文
- T3S: タスク固有特徴セレクタとスケジューラによるマルチタスク強化学習の改善マルチタスク強化学習
- 大規模マルチタスク世界モデルによる連続制御の学習マルチタスク強化学習
- 時間的に拡張された行動共有によるマルチタスク強化学習マルチタスク強化学習
- クロスタスク方策ガイダンスによる効率的なマルチタスク強化学習マルチタスク強化学習
- ロボティクス向け大規模並列マルチタスク強化学習のベンチマークマルチタスク強化学習
- 大きく、正則化し、カテゴリカルに:高容量価値関数は効率的なマルチタスク学習者マルチタスク強化学習
