PIVOT-R: プリミティブ駆動型ウェイポイント認識ワールドモデルによるロボットマニピュレーション
PIVOT-R: Primitive-Driven Waypoint-Aware World Model for Robotic Manipulation
言語指示に基づくロボット操作のため、プリミティブ動作の解析とウェイポイント予測を行うワールドモデルを提案し、非同期階層実行により効率を大幅に向上させた。
著者: Kaidong Zhang, Pengzhen Ren, Bingqian Lin, Junfan Lin, Shikui Ma, Hang Xu, Xiaodan Liang
分類: cs.RO, cs.AI, cs.CV, cs.LG
原文アブストラクト
Language-guided robotic manipulation is a challenging task that requires an embodied agent to follow abstract user instructions to accomplish various complex manipulation tasks. Previous work trivially fitting the data without revealing the relation between instruction and low-level executable actions, these models are prone to memorizing the surficial pattern of the data instead of acquiring the transferable knowledge, and thus are fragile to dynamic environment changes. To address this issue, we propose a PrIrmitive-driVen waypOinT-aware world model for Robotic manipulation (PIVOT-R) that focuses solely on the prediction of task-relevant waypoints. Specifically, PIVOT-R consists of a Waypoint-aware World Model (WAWM) and a lightweight action prediction module. The former performs primitive action parsing and primitive-driven waypoint prediction, while the latter focuses on decoding low-level actions. Additionally, we also design an asynchronous hierarchical executor (AHE), which can use different execution frequencies for different modules of the model, thereby helping the model reduce computational redundancy and improve model execution efficiency. Our PIVOT-R outperforms state-of-the-art (SoTA) open-source models on the SeaWave benchmark, achieving an average relative improvement of 19.45% across four levels of instruction tasks. Moreover, compared to the synchronously executed PIVOT-R, the execution efficiency of PIVOT-R with AHE is increased by 28-fold, with only a 2.9% drop in performance. These results provide compelling evidence that our PIVOT-R can significantly improve both the performance and efficiency of robotic manipulation.
関連論文
- 経験時代における実世界コード・アズ・ポリシー:ACE-CaP-0マニピュレーション
- 把持のアドミタンスによる入出力特性評価:設計パラメータと把持挙動の接続マニピュレーション
- ManiLadder:カテゴリ別・多難易度タスクラダーによるロボットマニピュレーションのベンチマークマニピュレーション
- 成功確率を変える事象を記憶する:Criticゲート付きイベントメモリによる長期的マニピュレーションマニピュレーション
- BiPOP: 長期的な両腕ロボット操作のための部分順序計画マニピュレーション
- RUMI: 拡張現実と触覚フィードバックを備えたRobotiq対応の汎用マニピュレーションインターフェースマニピュレーション
