単一の人間動画とオープンワールド物体グラフからの視覚ベースマニピュレーション
Vision-based Manipulation from Single Human Video with Open-World Object Graphs
1本の人間のRGB/RGB-D動画から物体中心の操作計画を抽出し、それを条件とする方策を学習することで、未知物体や背景・視点の変化に対応できる模倣学習手法ORIONを提案した。
著者: Yifeng Zhu, Arisrei Lim, Peter Stone, Yuke Zhu
分類: cs.RO, cs.CV, cs.LG
原文アブストラクト
This work presents an object-centric approach to learning vision-based manipulation skills from human videos. We investigate the problem of robot manipulation via imitation in the open-world setting, where a robot learns to manipulate novel objects from a single video demonstration. We introduce ORION, an algorithm that tackles the problem by extracting an object-centric manipulation plan from a single RGB or RGB-D video and deriving a policy that conditions on the extracted plan. Our method enables the robot to learn from videos captured by daily mobile devices and to generalize the policies to deployment environments with varying visual backgrounds, camera angles, spatial layouts, and novel object instances. We systematically evaluate our method on both short-horizon and long-horizon tasks, using RGB-D and RGB-only demonstration videos. Across varied tasks and demonstration types (RGB-D / RGB), we observe an average success rate of 74.4%, demonstrating the efficacy of ORION in learning from a single human video in the open world. Additional materials can be found on our project website: https://ut-austin-rpl.github.io/ORION-release.
