パラメータ化記号抽象グラフによるワンショット動画模倣
One-shot Video Imitation via Parameterized Symbolic Abstraction Graphs
動画デモを物体と関係性を表すパラメータ化記号抽象グラフとして解釈し、シミュレーションで力などの不可視属性を推定することで、新しい物体への汎化を実現したロボット操作手法。
著者: Jianren Wang, Kangni Liu, Dingkun Guo, Xian Zhou, Christopher G Atkeson
分類: cs.RO, cs.CV
原文アブストラクト
Learning to manipulate dynamic and deformable objects from a single demonstration video holds great promise in terms of scalability. Previous approaches have predominantly focused on either replaying object relationships or actor trajectories. The former often struggles to generalize across diverse tasks, while the latter suffers from data inefficiency. Moreover, both methodologies encounter challenges in capturing invisible physical attributes, such as forces. In this paper, we propose to interpret video demonstrations through Parameterized Symbolic Abstraction Graphs (PSAG), where nodes represent objects and edges denote relationships between objects. We further ground geometric constraints through simulation to estimate non-geometric, visually imperceptible attributes. The augmented PSAG is then applied in real robot experiments. Our approach has been validated across a range of tasks, such as Cutting Avocado, Cutting Vegetable, Pouring Liquid, Rolling Dough, and Slicing Pizza. We demonstrate successful generalization to novel objects with distinct visual and physical properties.
