四足歩行のためのSim-to-Real同時転移を実現する統合ロコモーションTransformer
Unified Locomotion Transformer with Simultaneous Sim-to-Real Transfer for Quadrupeds
特権情報を活用しつつ知識転移と方策最適化を単一のTransformerネットワークで統合し、1段階の学習でゼロショット実機展開を可能にした四足歩行制御フレームワークを提案。
著者: Dikai Liu, Tianwei Zhang, Jianxiong Yin, Simon See
分類: cs.RO, cs.LG
原文アブストラクト
Quadrupeds have gained rapid advancement in their capability of traversing across complex terrains. The adoption of deep Reinforcement Learning (RL), transformers and various knowledge transfer techniques can greatly reduce the sim-to-real gap. However, the classical teacher-student framework commonly used in existing locomotion policies requires a pre-trained teacher and leverages the privilege information to guide the student policy. With the implementation of large-scale models in robotics controllers, especially transformers-based ones, this knowledge distillation technique starts to show its weakness in efficiency, due to the requirement of multiple supervised stages. In this paper, we propose Unified Locomotion Transformer (ULT), a new transformer-based framework to unify the processes of knowledge transfer and policy optimization in a single network while still taking advantage of privilege information. The policies are optimized with reinforcement learning, next state-action prediction, and action imitation, all in just one training stage, to achieve zero-shot deployment. Evaluation results demonstrate that with ULT, optimal teacher and student policies can be obtained at the same time, greatly easing the difficulty in knowledge transfer, even with complex transformer-based models.
