日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

マニピュレーションarXiv:2405.15916

汎用事前学習済みViTを物体中心シーンエンコーダとしてマニピュレーション方策に転用

Recasting Generic Pretrained Vision Transformers As Object-Centric Scene Encoders For Manipulation Policies

シェア:XThreadsFacebookLINEはてブBluesky

汎用の事前学習済みVision Transformerのアテンションから物体らしい領域を切り出し、追加学習なしで物体中心の埋め込み表現を生成するSOFTを提案し、マニピュレーション方策の性能を大幅に向上させた。

著者: Jianing Qian, Anastasios Panagopoulos, Dinesh Jayaraman

分類: cs.CV, cs.RO

原文アブストラクト

Generic re-usable pre-trained image representation encoders have become a standard component of methods for many computer vision tasks. As visual representations for robots however, their utility has been limited, leading to a recent wave of efforts to pre-train robotics-specific image encoders that are better suited to robotic tasks than their generic counterparts. We propose Scene Objects From Transformers, abbreviated as SOFT, a wrapper around pre-trained vision transformer (PVT) models that bridges this gap without any further training. Rather than construct representations out of only the final layer activations, SOFT individuates and locates object-like entities from PVT attentions, and describes them with PVT activations, producing an object-centric embedding. Across standard choices of generic pre-trained vision transformers PVT, we demonstrate in each case that policies trained on SOFT(PVT) far outstrip standard PVT representations for manipulation tasks in simulated and real settings, approaching the state-of-the-art robotics-aware representations. Code, appendix and videos: https://sites.google.com/view/robot-soft/

関連論文

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2newmoとKDDIが資本業務提携 2028年に堺市で自動運転タクシー商用化目指す提携・買収
  3. 3NVIDIA、CoRLでGR00T 2とIsaac Lab 3.0を発表新製品・発表
  4. 4GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開
  5. 5Google DeepMind、ロボット向けAIを発表新製品・発表

PR・アフィリエイト広告

FRONTIER(GPU搭載BTOパソコン)

FRONTIER(GPU搭載BTOパソコン)の公式広告バナーFRONTIERでGPU搭載PCを見る ↗
PR本紙発行元 EmplifAI