日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

VLAarXiv:2410.01971

実行時観察介入で視覚言語行動モデルの視覚的堅牢性を向上

Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robust

シェア:XThreadsFacebookLINEはてブBluesky

VLAモデルが苦手とする妨害物体や背景色などのタスク無関係な視覚的詳細に対し、実行時に感度の高い領域を特定し自動画像編集で最小限の修正を加えるBYOVLAを提案。ファインチューニング不要で成功率低下を防ぐ。

著者: Asher J. Hancock, Allen Z. Ren, Anirudha Majumdar

分類: cs.RO, cs.LG

原文アブストラクト

Vision-language-action (VLA) models trained on large-scale internet data and robot demonstrations have the potential to serve as generalist robot policies. However, despite their large-scale training, VLAs are often brittle to task-irrelevant visual details such as distractor objects or background colors. We introduce Bring Your Own VLA (BYOVLA): a run-time intervention scheme that (1) dynamically identifies regions of the input image that the model is sensitive to, and (2) minimally alters task-irrelevant regions to reduce the model's sensitivity using automated image editing tools. Our approach is compatible with any off the shelf VLA without model fine-tuning or access to the model's weights. Hardware experiments on language-instructed manipulation tasks demonstrate that BYOVLA enables state-of-the-art VLA models to nearly retain their nominal performance in the presence of distractor objects and backgrounds, which otherwise degrade task success rates by up to 40%. Website with additional information, videos, and code: https://aasherh.github.io/byovla/ .

関連論文

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2newmoとKDDIが資本業務提携 2028年に堺市で自動運転タクシー商用化目指す提携・買収
  3. 3GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開
  4. 4NVIDIA、CoRLでGR00T 2とIsaac Lab 3.0を発表新製品・発表
  5. 5UC BerkeleyとNVIDIAの研究者、AIエージェントでロボット制御コードを仮想環境で改善研究・技術

PR・アフィリエイト広告

FRONTIER(GPU搭載BTOパソコン)

FRONTIER(GPU搭載BTOパソコン)の公式広告バナーFRONTIERでGPU搭載PCを見る ↗
PR本紙発行元 EmplifAI