LM-Nav: 大規模事前学習モデルによる言語・視覚・行動を統合したロボットナビゲーション
LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action
言語・視覚・行動の大規模事前学習モデル（GPT-3、CLIP、ViNG）を組み合わせ、言語指示による実世界の長距離ナビゲーションを追加学習なしで実現した。
著者: Dhruv Shah, Blazej Osinski, Brian Ichter, Sergey Levine
分類: cs.RO, cs.AI, cs.CL, cs.LG
原文アブストラクト
Goal-conditioned policies for robotic navigation can be trained on large, unannotated datasets, providing for good generalization to real-world settings. However, particularly in vision-based settings where specifying goals requires an image, this makes for an unnatural interface. Language provides a more convenient modality for communication with robots, but contemporary methods typically require expensive supervision, in the form of trajectories annotated with language descriptions. We present a system, LM-Nav, for robotic navigation that enjoys the benefits of training on unannotated large datasets of trajectories, while still providing a high-level interface to the user. Instead of utilizing a labeled instruction following dataset, we show that such a system can be constructed entirely out of pre-trained models for navigation (ViNG), image-language association (CLIP), and language modeling (GPT-3), without requiring any fine-tuning or language-annotated robot data. We instantiate LM-Nav on a real-world mobile robot and demonstrate long-horizon navigation through complex, outdoor environments from natural language instructions. For videos of our experiments, code release, and an interactive Colab notebook that runs in your browser, please check out our project page https://sites.google.com/view/lmnav
