移動運動学習における環境設計の重要性
Learning to Locomote: Understanding How Environment Design Matters for Deep Reinforcement Learning
深層強化学習で移動運動を学習する際、状態表現や報酬構造、制御周波数などの環境設計の選択が結果の脆さに大きく影響することを示した論文。
著者: Daniele Reda, Tianxin Tao, Michiel van de Panne
分類: cs.LG, cs.RO
原文アブストラクト
Learning to locomote is one of the most common tasks in physics-based animation and deep reinforcement learning (RL). A learned policy is the product of the problem to be solved, as embodied by the RL environment, and the RL algorithm. While enormous attention has been devoted to RL algorithms, much less is known about the impact of design choices for the RL environment. In this paper, we show that environment design matters in significant ways and document how it can contribute to the brittle nature of many RL results. Specifically, we examine choices related to state representations, initial state distributions, reward structure, control frequency, episode termination procedures, curriculum usage, the action space, and the torque limits. We aim to stimulate discussion around such choices, which in practice strongly impact the success of RL when applied to continuous-action control problems of interest to animation, such as learning to locomote.
