Astra: 身体性指示追従のための効率的Transformerアーキテクチャと対照的ダイナミクス学習
Astra: Efficient Transformer Architecture and Contrastive Dynamics Learning for Embodied Instruction Following
異なるモダリティのセグメントからなる軌跡を効率的に処理する新しいTransformerと、環境ダイナミクス理解を深める対照的学習目的を提案し、ロボット操作ベンチマークで性能を大幅に向上させた。
著者: Yueen Ma, Dafeng Chi, Shiguang Wu, Yuecheng Liu, Yuzheng Zhuang, Irwin King
分類: cs.RO
原文アブストラクト
Vision-language-action models have gained significant attention for their ability to model multimodal sequences in embodied instruction following tasks. However, most existing models rely on causal attention, which we find suboptimal for processing sequences composed of interleaved segments from different modalities. In this paper, we introduce Astra, a novel Transformer architecture featuring trajectory attention and learnable action queries, designed to efficiently process segmented multimodal trajectories and predict actions for imitation learning. Furthermore, we propose a contrastive dynamics learning objective to enhance the model's understanding of environment dynamics and multimodal alignment, complementing the primary behavior cloning objective. Through extensive experiments on three large-scale robot manipulation benchmarks, Astra demonstrates substantial performance improvements over previous models.
