ピクセルベース階層型ポリシーによるタスク汎化の利点
On the benefits of pixel-based hierarchical policies for task generalization
画像入力のロボット制御において、階層型強化学習ポリシーがタスク間の汎化性能を高め、微調整の手間を減らすことを実験的に示した。
著者: Tudor Cristea-Platon, Bogdan Mazoure, Josh Susskind, Walter Talbott
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
Reinforcement learning practitioners often avoid hierarchical policies, especially in image-based observation spaces. Typically, the single-task performance improvement over flat-policy counterparts does not justify the additional complexity associated with implementing a hierarchy. However, by introducing multiple decision-making levels, hierarchical policies can compose lower-level policies to more effectively generalize between tasks, highlighting the need for multi-task evaluations. We analyze the benefits of hierarchy through simulated multi-task robotic control experiments from pixels. Our results show that hierarchical policies trained with task conditioning can (1) increase performance on training tasks, (2) lead to improved reward and state-space generalizations in similar tasks, and (3) decrease the complexity of fine tuning required to solve novel tasks. Thus, we believe that hierarchical policies should be considered when building reinforcement learning architectures capable of generalizing between tasks.
