LeLaN: 実世界動画から言語条件付きナビゲーション方策を学習
LeLaN: Learning A Language-Conditioned Navigation Policy from In-the-Wild Videos
大規模視覚言語モデルとロボット基盤モデルを活用し、ラベルなし・行動なしの一人称視点動画から言語で指示された物体ナビゲーション方策を学習する手法を提案。1000回以上の実世界実験で既存手法を上回り、エッジ計算で4倍高速に推論可能。
著者: Noriaki Hirose, Catherine Glossop, Ajay Sridhar, Dhruv Shah, Oier Mees, Sergey Levine
分類: cs.RO
原文アブストラクト
The world is filled with a wide variety of objects. For robots to be useful, they need the ability to find arbitrary objects described by people. In this paper, we present LeLaN(Learning Language-conditioned Navigation policy), a novel approach that consumes unlabeled, action-free egocentric data to learn scalable, language-conditioned object navigation. Our framework, LeLaN leverages the semantic knowledge of large vision-language models, as well as robotic foundation models, to label in-the-wild data from a variety of indoor and outdoor environments. We label over 130 hours of data collected in real-world indoor and outdoor environments, including robot observations, YouTube video tours, and human walking data. Extensive experiments with over 1000 real-world trials show that our approach enables training a policy from unlabeled action-free videos that outperforms state-of-the-art robot navigation methods, while being capable of inference at 4 times their speed on edge compute. We open-source our models, datasets and provide supplementary videos on our project page (https://learning-language-navigation.github.io/).
