PRISM-DP: セグメンテーション・メッシュ生成・姿勢追跡による拡散ポリシーのための空間姿勢観測
PRISM-DP: Spatial Pose-based Observations for Diffusion-Policies via Segmentation, Mesh Generation, and Pose Tracking
セグメンテーションとメッシュ生成、姿勢追跡を組み合わせ、手動メッシュ作成なしで物体の空間姿勢を観測として拡散ポリシーを学習する手法を提案。画像ベースより高精度で、真の状態と同等の性能を実現。
著者: Xiatao Sun, Yinxing Chen, Daniel Rakita
分類: cs.RO
原文アブストラクト
Diffusion policies generate robot motions by learning to denoise action-space trajectories conditioned on observations. These observations are commonly streams of RGB images, whose high dimensionality includes substantial task-irrelevant information, requiring large models to extract relevant patterns. In contrast, using structured observations like the spatial poses of key objects enables training more compact policies with fewer parameters. However, obtaining accurate object poses in open-set, real-world environments remains challenging, as 6D pose estimation and tracking methods often depend on markers placed on objects beforehand or pre-scanned object meshes that require manual reconstruction. We propose PRISM-DP, an approach that leverages segmentation, mesh generation, and pose tracking models to enable compact diffusion policy learning directly from the spatial poses of task-relevant objects. Crucially, by using a mesh generation model, PRISM-DP eliminates the need for manual mesh creation, improving scalability in open-set environments. Experiments in simulation and the real world show that PRISM-DP outperforms high-dimensional image-based policies and achieves performance comparable to policies trained with ground-truth state information.
