SYNERGAI: 人間とロボットの協調のための知覚アライメント
SYNERGAI: Perception Alignment for Human-Robot Collaboration
3Dシーングラフを介して人間とロボットの知覚のずれをオンライン対話で自動修正し、協調作業を実現するLLMベースのシステムを提案した。
著者: Yixin Chen, Guoxi Zhang, Yaowei Zhang, Hongming Xu, Peiyuan Zhi, Qing Li, Siyuan Huang
分類: cs.RO
原文アブストラクト
Recently, large language models (LLMs) have shown strong potential in facilitating human-robotic interaction and collaboration. However, existing LLM-based systems often overlook the misalignment between human and robot perceptions, which hinders their effective communication and real-world robot deployment. To address this issue, we introduce SYNERGAI, a unified system designed to achieve both perceptual alignment and human-robot collaboration. At its core, SYNERGAI employs 3D Scene Graph (3DSG) as its explicit and innate representation. This enables the system to leverage LLM to break down complex tasks and allocate appropriate tools in intermediate steps to extract relevant information from the 3DSG, modify its structure, or generate responses. Importantly, SYNERGAI incorporates an automatic mechanism that enables perceptual misalignment correction with users by updating its 3DSG with online interaction. SYNERGAI achieves comparable performance with the data-driven models in ScanQA in a zero-shot manner. Through comprehensive experiments across 10 real-world scenes, SYNERGAI demonstrates its effectiveness in establishing common ground with humans, realizing a success rate of 61.9% in alignment tasks. It also significantly improves the success rate from 3.7% to 45.68% on novel tasks by transferring the knowledge acquired during alignment.
関連論文
- 誤った例を用いたロボットポリシーの人間への教授法Human-Robot Collaboration
- リサイクル選別における人とロボットの協働:人間の関与度が作業性能と円滑さに与える影響Human-Robot Collaboration
