ボトムアップ型メタ方策探索
Bottom-Up Meta-Policy Search
少数の専門家方策から未知タスクに汎化するメタ方策を蒸留し、方策フィルタリングで素早く適応するメタ学習アルゴリズムを提案し、RoboCup 3Dサッカーでキック動作学習に適用した。
著者: Luckeciano C. Melo, Marcos R. O. A. Maximo, Adilson Marques da Cunha
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
Despite of the recent progress in agents that learn through interaction, there are several challenges in terms of sample efficiency and generalization across unseen behaviors during training. To mitigate these problems, we propose and apply a first-order Meta-Learning algorithm called Bottom-Up Meta-Policy Search (BUMPS), which works with two-phase optimization procedure: firstly, in a meta-training phase, it distills few expert policies to create a meta-policy capable of generalizing knowledge to unseen tasks during training; secondly, it applies a fast adaptation strategy named Policy Filtering, which evaluates few policies sampled from the meta-policy distribution and selects which best solves the task. We conducted all experiments in the RoboCup 3D Soccer Simulation domain, in the context of kick motion learning. We show that, given our experimental setup, BUMPS works in scenarios where simple multi-task Reinforcement Learning does not. Finally, we performed experiments in a way to evaluate each component of the algorithm.
