アシストロボットのための言語粒度とオンザフライ制御
LLM Granularity for On-the-Fly Robot Control
視覚が使えない状況でも言語のみでアシストロボットを制御できるか検討し、異なる粒度の言語指示への応答とオンザフライ制御の必要性・実現可能性をSawyerとTurtlebotで実験した。
著者: Peng Wang, Mattia Robbiani, Zhihao Guo
分類: cs.RO, cs.AI
原文アブストラクト
Assistive robots have attracted significant attention due to their potential to enhance the quality of life for vulnerable individuals like the elderly. The convergence of computer vision, large language models, and robotics has introduced the `visuolinguomotor' mode for assistive robots, where visuals and linguistics are incorporated into assistive robots to enable proactive and interactive assistance. This raises the question: \textit{In circumstances where visuals become unreliable or unavailable, can we rely solely on language to control robots, i.e., the viability of the `linguomotor` mode for assistive robots?} This work takes the initial steps to answer this question by: 1) evaluating the responses of assistive robots to language prompts of varying granularities; and 2) exploring the necessity and feasibility of controlling the robot on-the-fly. We have designed and conducted experiments on a Sawyer cobot to support our arguments. A Turtlebot robot case is designed to demonstrate the adaptation of the solution to scenarios where assistive robots need to maneuver to assist. Codes will be released on GitHub soon to benefit the community.
