長期マルチゴール強化学習のための最大エントロピー獲得探索
Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learning
到達困難な目標を追わず、達成済み目標分布のエントロピーを最大化する自己目標を設定することで、迷路探索やブロック積みなどの長期タスクで探索効率を大幅に向上させた研究。
著者: Silviu Pitis, Harris Chan, Stephen Zhao, Bradly Stadie, Jimmy Ba
分類: cs.LG, cs.AI, cs.RO, stat.ML
原文アブストラクト
What goals should a multi-goal reinforcement learning agent pursue during training in long-horizon tasks? When the desired (test time) goal distribution is too distant to offer a useful learning signal, we argue that the agent should not pursue unobtainable goals. Instead, it should set its own intrinsic goals that maximize the entropy of the historical achieved goal distribution. We propose to optimize this objective by having the agent pursue past achieved goals in sparsely explored areas of the goal space, which focuses exploration on the frontier of the achievable goal set. We show that our strategy achieves an order of magnitude better sample efficiency than the prior state of the art on long-horizon multi-goal tasks including maze navigation and block stacking.
