視覚-言語-行動モデルにおける特徴の観測と制御
Observing and Controlling Features in Vision-Language-Action Models
VLAモデルの内部表現を線形分類器で観測し、最適制御に基づく軽量な介入で出力を望む方向に操る手法を提案。微調整なしでロボットの行動をリアルタイムに調整できることを示した。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Hugo Buurmeijer, Carmen Amo Alonso, Aiden Swann, Marco Pavone
分類: cs.RO
原文アブストラクト
Vision-Language-Action Models (VLAs) have shown remarkable progress towards embodied intelligence. While their architecture partially resembles that of Large Language Models (LLMs), VLAs exhibit higher complexity due to their multi-modal inputs/outputs and often hybrid nature of transformer and diffusion heads. This is part of the reason why insights from mechanistic interpretability in LLMs, which explain how the internal model representations relate to their output behavior, do not trivially transfer to VLA counterparts. In this work, we propose to close this gap by introducing and analyzing two main concepts: feature-observability and feature-controllability. In particular, we first study features that are linearly encoded in representation space, and show how they can be observed by means of a linear classifier. Then, we use a minimal linear intervention grounded in optimal control to accurately place internal representations and steer the VLA's output towards a desired region. Our results show that targeted, lightweight interventions can reliably steer a robot's behavior while preserving closed-loop capabilities. We demonstrate on different VLA architectures ($\pi_{0.5}$ and OpenVLA) through simulation experiments that VLAs possess interpretable internal structure amenable to online adaptation without fine-tuning, enabling real-time alignment with user preferences and task requirements.
関連論文
- 視覚・言語・行動モデルの潜在クラスタ解析VLA/解釈可能性
- VLA表現からのタスク進捗の解読VLA/解釈可能性
- 具現化解釈可能性：視覚-言語-行動モデルにおける因果理解と汎化の関連付けVLA/解釈可能性
- スパースオートエンコーダがVLAモデルの解釈可能で操作可能な特徴を明らかにするVLA/解釈可能性
