少ないデータで歩行を学ぶ：四足歩行のためのDynaスタイルアプローチ
Learning to Walk With Less: A Dyna-Style Approach to Quadrupedal Locomotion
モデルベースのDynaスタイル枠組みでPPOのロールアウトに合成データを追加し、四足歩行制御のサンプル効率を改善した。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Francisco Affonso, Felipe Tommaselli, João H. Aléssio, Vivian S. Medeiros, Mateus V. Gasparino, Girish Chowdhary, Marcelo Becker
分類: cs.RO, cs.AI
原文アブストラクト
Traditional on-policy reinforcement learning (RL) controllers for quadrupedal locomotion often suffer from low data efficiency, requiring millions of interactions with simulated environments to achieve stable control. We integrate model-based techniques that improve sample efficiency by augmenting PPO rollouts with synthetic data in a Dyna-style framework. Our method employs a learned transition model to generate short-horizon synthetic tails for each trajectory, anchored by physics-based simulation to preserve stability. A predefined scheduling strategy gradually integrates synthetic transitions, preventing model usage during early training stages when prediction accuracy is low. Through extensive ablation studies, we analyze how varying data parameters influence PPO's learning behavior. Finally, we validate our method in simulation on a Unitree Go1 robot, reaching convergence with substantially fewer simulation steps (19.64M vs. 27.53M) and a 12.24% reduction in wall-clock training time, without compromising policy performance or convergence. Cross-platform experiments on ANYmal and Unitree Go2 further confirm the framework's ability to learn high-dimensional locomotion control with substantially reduced simulation experience, despite reward trade-offs on complex morphologies.
