MoRE: 四足歩行ロボットの視覚-言語-行動モデルにおける強化学習のスケーラビリティを解き放つ
MoRE: Unlocking Scalability in Reinforcement Learning for Quadruped Vision-Language-Action Models
四足歩行ロボット向けに、複数の低ランク適応モジュールを専門家として組み込んだスパース活性化の混合エキスパート型VLAモデルを提案し、強化学習で混合品質データから効率的に学習することで多様なタスク性能と汎化性を実現した。
著者: Han Zhao, Wenxuan Song, Donglin Wang, Xinyang Tong, Pengxiang Ding, Xuelian Cheng, Zongyuan Ge
分類: cs.RO, cs.AI
原文アブストラクト
Developing versatile quadruped robots that can smoothly perform various actions and tasks in real-world environments remains a significant challenge. This paper introduces a novel vision-language-action (VLA) model, mixture of robotic experts (MoRE), for quadruped robots that aim to introduce reinforcement learning (RL) for fine-tuning large-scale VLA models with a large amount of mixed-quality data. MoRE integrates multiple low-rank adaptation modules as distinct experts within a dense multi-modal large language model (MLLM), forming a sparse-activated mixture-of-experts model. This design enables the model to effectively adapt to a wide array of downstream tasks. Moreover, we employ a reinforcement learning-based training objective to train our model as a Q-function after deeply exploring the structural properties of our tasks. Effective learning from automatically collected mixed-quality data enhances data efficiency and model performance. Extensive experiments demonstrate that MoRE outperforms all baselines across six different skills and exhibits superior generalization capabilities in out-of-distribution scenarios. We further validate our method in real-world scenarios, confirming the practicality of our approach and laying a solid foundation for future research on multi-task learning in quadruped robots.
