Spot上での高性能強化学習:分布尺度を用いたシミュレーションパラメータ最適化
High-Performance Reinforcement Learning on Spot: Optimizing Simulation Parameters with Distributional Measures
Spotの低レベルAPI向けに強化学習ポリシーを訓練・実機展開し、Wasserstein距離とMMDでsim2realギャップを定量化してCMA-ESでシミュレーションパラメータを最適化。5.2m/s超の走行や滑り面耐性を実現。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: AJ Miller, Fangzhou Yu, Michael Brauckmann, Farbod Farshidian
分類: cs.LG, cs.RO
原文アブストラクト
This work presents an overview of the technical details behind a high performance reinforcement learning policy deployment with the Spot RL Researcher Development Kit for low level motor access on Boston Dynamics Spot. This represents the first public demonstration of an end to end end reinforcement learning policy deployed on Spot hardware with training code publicly available through Nvidia IsaacLab and deployment code available through Boston Dynamics. We utilize Wasserstein Distance and Maximum Mean Discrepancy to quantify the distributional dissimilarity of data collected on hardware and in simulation to measure our sim2real gap. We use these measures as a scoring function for the Covariance Matrix Adaptation Evolution Strategy to optimize simulated parameters that are unknown or difficult to measure from Spot. Our procedure for modeling and training produces high quality reinforcement learning policies capable of multiple gaits, including a flight phase. We deploy policies capable of over 5.2ms locomotion, more than triple Spots default controller maximum speed, robustness to slippery surfaces, disturbance rejection, and overall agility previously unseen on Spot. We detail our method and release our code to support future work on Spot with the low level API.
