RoboInspector:LLMロボット操作におけるポリシーコードの信頼性欠如を暴く
RoboInspector: Unveiling the Unreliability of Policy Code for LLM-enabled Robotic Manipulation
LLMによるロボット操作のポリシーコード生成が、タスクの複雑さや指示の粒度によって不安定になる問題を体系的に分析し、失敗原因を4つに分類。失敗コードのフィードバックによる改善手法で信頼性を最大35%向上させた。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Chenduo Ying, Linkang Du, Peng Cheng, Yuanchao Shu
分類: cs.RO, cs.AI
原文アブストラクト
Large language models (LLMs) demonstrate remarkable capabilities in reasoning and code generation, enabling robotic manipulation to be initiated with just a single instruction. The LLM carries out various tasks by generating policy code required to control the robot. Despite advances in LLMs, achieving reliable policy code generation remains a significant challenge due to the diverse requirements of real-world tasks and the inherent complexity of user instructions. In practice, different users may provide distinct instructions to drive the robot for the same task, which may cause the unreliability of policy code generation. To bridge this gap, we design \textbf{RoboInspector}, a pipeline to unveil and characterize the unreliability of the policy code for LLM-enabled robotic manipulation from two perspectives: the complexity of the manipulation task and the granularity of the instruction. We perform comprehensive experiments with 216 distinct combinations of tasks, instructions, and LLMs in two prominent frameworks. The \textbf{RoboInspector} identifies four main unreliable behaviors that lead to manipulation failure. We provide a detailed characterization of these behaviors and their underlying causes, giving insight for practical development to reduce unreliability. Furthermore, we introduce a refinement approach guided by failure policy code feedback that improves the reliability of policy code generation by up to 35\% in LLM-enabled robotic manipulation, evaluated in both simulation and real-world environments.
