意味のあるタスク空間を弱教師あり学習で切り出す強化学習
Weakly-Supervised Reinforcement Learning for Controllable Behavior
弱い監督信号を用いて、無意味なタスクの巨大な空間から意味的に整合したタスク部分空間を自動的に分離する枠組みを提案し、探索効率と状態間距離の表現獲得を改善した。
著者: Lisa Lee, Benjamin Eysenbach, Ruslan Salakhutdinov, Shixiang Shane Gu, Chelsea Finn
分類: cs.LG, cs.RO, stat.ML
原文アブストラクト
Reinforcement learning (RL) is a powerful framework for learning to take actions to solve tasks. However, in many settings, an agent must winnow down the inconceivably large space of all possible tasks to the single task that it is currently being asked to solve. Can we instead constrain the space of tasks to those that are semantically meaningful? In this work, we introduce a framework for using weak supervision to automatically disentangle this semantically meaningful subspace of tasks from the enormous space of nonsensical "chaff" tasks. We show that this learned subspace enables efficient exploration and provides a representation that captures distance between states. On a variety of challenging, vision-based continuous control problems, our approach leads to substantial performance gains, particularly as the complexity of the environment grows.
