線形クープマンダイナミクスを用いた非線形ロボットシステムの効率的強化学習
Efficient Reinforcement Learning using Linear Koopman Dynamics for Nonlinear Robotic Systems
クープマン作用素理論で非線形システムの線形リフトダイナミクスを学習し、アクタークリティック型強化学習に統合することで、サンプル効率の高い閉ループ制御を実現した。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Wenjian Hao, Yuxuan Fang, Zehui Lu, Shaoshuai Mou
分類: cs.RO, cs.SY, eess.SY
原文アブストラクト
This paper presents a model-based reinforcement learning (RL) framework for optimal closed-loop control of nonlinear robotic systems. The proposed approach learns linear lifted dynamics through Koopman operator theory and integrates the resulting model into an actor-critic architecture for policy optimization, where the policy represents a parameterized closed-loop controller. To reduce computational cost and mitigate model rollout errors, policy gradients are estimated using one-step predictions of the learned dynamics rather than multi-step propagation. This leads to an online mini-batch policy gradient framework that enables policy improvement from streamed interaction data. The proposed framework is evaluated on several simulated nonlinear control benchmarks and two real-world hardware platforms, including a Kinova Gen3 robotic arm and a Unitree Go1 quadruped. Experimental results demonstrate improved sample efficiency over model-free RL baselines, superior control performance relative to model-based RL baselines, and control performance comparable to classical model-based methods that rely on exact system dynamics.
