特徴抽出器か意思決定者か：視覚運動ポリシーにおける視覚エンコーダの役割を再考する
Feature Extractor or Decision Maker: Rethinking the Role of Visual Encoders in Visuomotor Policies
視覚運動ポリシーにおいて視覚エンコーダが単なる特徴抽出器ではなく意思決定にも寄与していることを実験的に示し、OOD事前学習モデルの性能低下を明らかにした研究。
著者: Ruiyu Wang, Zheyu Zhuang, Shutong Jin, Nils Ingelhag, Danica Kragic, Florian T. Pokorny
分類: cs.RO
原文アブストラクト
An end-to-end (E2E) visuomotor policy is typically treated as a unified whole, but recent approaches using out-of-domain (OOD) data to pretrain the visual encoder have cleanly separated the visual encoder from the network, with the remainder referred to as the policy. We propose Visual Alignment Testing, an experimental framework designed to evaluate the validity of this functional separation. Our results indicate that in E2E-trained models, visual encoders actively contribute to decision-making resulting from motor data supervision, contradicting the assumed functional separation. In contrast, OOD-pretrained models, where encoders lack this capability, experience an average performance drop of 42\% in our benchmark results, compared to the state-of-the-art performance achieved by E2E policies. We believe this initial exploration of visual encoders' role can provide a first step towards guiding future pretraining methods to address their decision-making ability, such as developing task-conditioned or context-aware encoders.
