視覚言語モデルによる古典的タスクプランナの接地
Grounding Classical Task Planners via Vision-Language Models
古典的プランナが生成した行動を視覚言語モデルで検証し、行動失敗の検出とアフォーダンス確認を行うことで、実環境での計画実行を成功させるフレームワークTPVQAを提案した。
著者: Xiaohan Zhang, Yan Ding, Saeid Amiri, Hao Yang, Andy Kaminski, Chad Esselink, Shiqi Zhang
分類: cs.RO
原文アブストラクト
Classical planning systems have shown great advances in utilizing rule-based human knowledge to compute accurate plans for service robots, but they face challenges due to the strong assumptions of perfect perception and action executions. To tackle these challenges, one solution is to connect the symbolic states and actions generated by classical planners to the robot's sensory observations, thus closing the perception-action loop. This research proposes a visually-grounded planning framework, named TPVQA, which leverages Vision-Language Models (VLMs) to detect action failures and verify action affordances towards enabling successful plan execution. Results from quantitative experiments show that TPVQA surpasses competitive baselines from previous studies in task completion rate.
