CACTO-BIC: バイアス付きサンプリングとGPU加速軌道最適化によるスケーラブルなアクター・クリティック学習
CACTO-BIC: Scalable Actor-Critic Learning via Biased Sampling and GPU-Accelerated Trajectory Optimization
軌道最適化と強化学習を組み合わせたCACTOを改良し、価値関数に基づく初期状態サンプリングのバイアスとGPU加速により、サンプル効率と計算速度を向上させた。四足ロボットAlienGOでリアルタイム応用可能なスケーラビリティを示した。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Elisa Alboni, Pietro Noah Crestaz, Elias Fontanari, Andrea Del Prete
分類: cs.RO, math.OC
原文アブストラクト
Trajectory Optimization (TO) and Reinforcement Learning (RL) offer complementary strengths for solving optimal control problems. TO efficiently computes locally optimal solutions but can struggle with non-convexity, while RL is more robust to non-convexity at the cost of significantly higher computational demands. CACTO (Continuous Actor-Critic with Trajectory Optimization) was introduced to combine these advantages by learning a warm-start policy that guides the TO solver towards low-cost trajectories. However, scalability remains a key limitation, as increasing system complexity significantly raises the computational cost of TO. This work introduces CACTO-BIC to address these challenges. CACTO-BIC improves data efficiency by biasing initial-state sampling leveraging a property of the value function associated with locally optimal policies; moreover, it reduces computation time by exploiting GPU acceleration. Empirical evaluations show improved sample efficiency and faster computation compared to CACTO. Comparisons with PPO demonstrate that our approach can achieve similar solutions in less time. Finally, experiments on the AlienGO quadruped robot demonstrate that CACTO-BIC can scale to high-dimensional systems and is suitable for real-time applications.
関連論文
- Sobolev学習で軌道最適化を活用した連続アクター・クリティックの改善強化学習×軌道最適化
- 軌道最適化と深層強化学習の利点を組み合わせる強化学習×軌道最適化
