大規模言語モデルからゼロショット常識を抽出するロボット3Dシーン理解
Extracting Zero-shot Common Sense from Large Language Models for Robot 3D Scene Understanding
大規模言語モデルの常識知識を活用し、3Dシーングラフ内の物体から部屋ラベルをゼロショットで推定する手法を提案した。
著者: William Chen, Siyi Hu, Rajat Talak, Luca Carlone
分類: cs.RO, cs.CL
原文アブストラクト
Semantic 3D scene understanding is a problem of critical importance in robotics. While significant advances have been made in simultaneous localization and mapping algorithms, robots are still far from having the common sense knowledge about household objects and their locations of an average human. We introduce a novel method for leveraging common sense embedded within large language models for labelling rooms given the objects contained within. This algorithm has the added benefits of (i) requiring no task-specific pre-training (operating entirely in the zero-shot regime) and (ii) generalizing to arbitrary room and object labels, including previously-unseen ones -- both of which are highly desirable traits in robotic scene understanding algorithms. The proposed algorithm operates on 3D scene graphs produced by modern spatial perception systems, and we hope it will pave the way to more generalizable and scalable high-level 3D scene understanding for robotics.
