Words2Contact: 基盤モデルを用いた言語指示からの支持接触点同定
Words2Contact: Identifying Support Contacts from Verbal Instructions Using Foundation Models
大規模言語モデルと視覚言語モデルを活用し、人間の自然言語指示からロボットの支持接触位置を予測するパイプラインを提案し、実機ヒューマノイドで検証した。
著者: Dionis Totsila, Quentin Rouxel, Jean-Baptiste Mouret, Serena Ivaldi
分類: cs.RO, cs.AI
原文アブストラクト
This paper presents Words2Contact, a language-guided multi-contact placement pipeline leveraging large language models and vision language models. Our method is a key component for language-assisted teleoperation and human-robot cooperation, where human operators can instruct the robots where to place their support contacts before whole-body reaching or manipulation using natural language. Words2Contact transforms the verbal instructions of a human operator into contact placement predictions; it also deals with iterative corrections, until the human is satisfied with the contact location identified in the robot's field of view. We benchmark state-of-the-art LLMs and VLMs for size and performance in contact prediction. We demonstrate the effectiveness of the iterative correction process, showing that users, even naive, quickly learn how to instruct the system to obtain accurate locations. Finally, we validate Words2Contact in real-world experiments with the Talos humanoid robot, instructed by human operators to place support contacts on different locations and surfaces to avoid falling when reaching for distant objects.
