ニューラルモーションシミュレータ:強化学習における世界モデルの限界を押し広げる
Neural Motion Simulator: Pushing the Limit of World Models in Reinforcement Learning
身体システムの将来の物理状態を予測する世界モデルMoSimを提案し、想像世界での効率的なスキル獲得やゼロショット強化学習を可能にするとともに、任意のモデルフリーRLをモデルベース化してサンプル効率と汎化性能を向上させる。
著者: Chenjie Hao, Weyl Lu, Yifan Xu, Yubei Chen
分類: cs.LG, cs.RO
原文アブストラクト
An embodied system must not only model the patterns of the external world but also understand its own motion dynamics. A motion dynamic model is essential for efficient skill acquisition and effective planning. In this work, we introduce the neural motion simulator (MoSim), a world model that predicts the future physical state of an embodied system based on current observations and actions. MoSim achieves state-of-the-art performance in physical state prediction and provides competitive performance across a range of downstream tasks. This works shows that when a world model is accurate enough and performs precise long-horizon predictions, it can facilitate efficient skill acquisition in imagined worlds and even enable zero-shot reinforcement learning. Furthermore, MoSim can transform any model-free reinforcement learning (RL) algorithm into a model-based approach, effectively decoupling physical environment modeling from RL algorithm development. This separation allows for independent advancements in RL algorithms and world modeling, significantly improving sample efficiency and enhancing generalization capabilities. Our findings highlight that world models for motion dynamics is a promising direction for developing more versatile and capable embodied systems.
