3D-TAFS: 訓練不要の3Dアフォーダンスセグメンテーションフレームワーク
3D-TAFS: A Training-free Framework for 3D Affordance Segmentation
大規模マルチモーダルモデルと3D視覚ネットワークを組み合わせ、訓練なしで言語指示から3D物体のアフォーダンス領域を切り出す手法を提案し、屋内シーン向けベンチマークIndoorAfford-Benchも構築した。
著者: Meng Chu, Xuan Zhang, Zhedong Zheng, Tat-Seng Chua
分類: cs.RO
原文アブストラクト
Translating high-level linguistic instructions into precise robotic actions in the physical world remains challenging, particularly when considering the feasibility of interacting with 3D objects. In this paper, we introduce 3D-TAFS, a novel training-free multimodal framework for 3D affordance segmentation. To facilitate a comprehensive evaluation of such frameworks, we present IndoorAfford-Bench, a large-scale benchmark containing 9,248 images spanning 20 diverse indoor scenes across 6 areas, supporting standardized interaction queries. In particular, our framework integrates a large multimodal model with a specialized 3D vision network, enabling a seamless fusion of 2D and 3D visual understanding with language comprehension. Extensive experiments on IndoorAfford-Bench validate the proposed 3D-TAFS's capability in handling interactive 3D affordance segmentation tasks across diverse settings, showcasing competitive performance across various metrics. Our results highlight 3D-TAFS's potential for enhancing human-robot interaction based on affordance understanding in complex indoor environments, advancing the development of more intuitive and efficient robotic frameworks for real-world applications.
