日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

ナビゲーションarXiv:2506.09839

OctoNav: 汎用身体性ナビゲーションに向けて

OctoNav: Towards Generalist Embodied Navigation

シェア:XThreadsFacebookLINEはてブBluesky

マルチモーダルで多能力な自由形式指示に従える汎用ナビゲーションエージェントのための大規模ベンチマークOctoNav-Benchと、MLLMをVLA化したOctoNav-R1を提案。

詳しい要約

1. どんなもの?

- 汎用embodied navigation agentの実現を目指す研究 - 自由形式instructionで多modal・多capabilityの任意の組み合わせを扱う - 大規模benchmark「OctoNav-Bench」と手法「OctoNav-R1」を提案 - OctoNav-Benchは連続環境でannotation pipelineにより構築 - instruction-trajectory pairを多様なmodality/capabilityで作成 - 行動の思考過程を与えるThink-Before-Action (TBA-CoT) datasetも構築

2. 先行研究と比べてどこがすごい?

- 従来のnavigation研究はObjNav, ImgNav, VLNなどtask/capabilityごとに分割 - 各taskで目的・modalityが異なり、datasetと手法が個別設計されていた - 本研究はこれらを統合しgeneralist navigation agentを目指す点が新しい - 自由形式instructionで任意のmulti-modal/multi-capabilityの組み合わせを扱う - 単一task benchmarkではなく統合benchmarkと手法を同時提案

3. 技術・手法の肝は?

- OctoNav-R1はMLLMを基盤にVLA-type modelへ適応 - 2D visual observationのみからlow-level actionを生成 - Hybrid Training Paradigm (HTP)を設計、3段階から構成 - Action-/TBA-SFT stage - Nav-GPRO stage - Online RL stage - 各stageに学習policyとrewardを設計 - TBA-SFTとNav-GRPOはOpenAI-o1やDeepSeek-R1のthinking-before-answerに着想 - TBA-SFTでTBA-CoT datasetを用いcold-startとしてfine-tune - 続いてNav-GPROでthinking abilityを改善

4. どうやって有効だと検証した?

- OctoNav-R1を先行手法と比較し優れた性能を示すと報告 - OctoNav-Bench上での評価を実施(詳細な実験設定は要旨からは不明) - 具体的なmetric・baseline・ablationの内容は要旨からは不明

5. 議論はある?

- embodied navigation分野でthinking-before-actionをどう実現するかを探究 - generalistへ向けたmodelのreasoning ability向上を狙う - 各stageのlearning policyとreward設計の詳細は要旨からは不明 - 限界やfailure case、計算コスト等の議論は要旨からは不明

6. 次に読むべき論文は?

- OpenAI-o1 (thinking-before-answerの着想元) - DeepSeek-R1 (reasoning abilityの着想元) - ObjNav, ImgNav, VLNの各先行navigation研究 - MLLMおよびVLA-type modelの関連研究 - GRPO系のRL手法(Nav-GPROの基礎)

※ AIが要旨から生成した要約です。正確性は原文をご確認ください。

著者: Chen Gao, Liankai Jin, Xingyu Peng, Jiazhao Zhang, Yue Deng, Annan Li, He Wang, Si Liu

分類: cs.CV, cs.AI, cs.RO

原文アブストラクト

Embodied navigation stands as a foundation pillar within the broader pursuit of embodied AI. However, previous navigation research is divided into different tasks/capabilities, e.g., ObjNav, ImgNav and VLN, where they differ in task objectives and modalities, making datasets and methods are designed individually. In this work, we take steps toward generalist navigation agents, which can follow free-form instructions that include arbitrary compounds of multi-modal and multi-capability. To achieve this, we propose a large-scale benchmark and corresponding method, termed OctoNav-Bench and OctoNav-R1. Specifically, OctoNav-Bench features continuous environments and is constructed via a designed annotation pipeline. We thoroughly craft instruction-trajectory pairs, where instructions are diverse in free-form with arbitrary modality and capability. Also, we construct a Think-Before-Action (TBA-CoT) dataset within OctoNav-Bench to provide the thinking process behind actions. For OctoNav-R1, we build it upon MLLMs and adapt it to a VLA-type model, which can produce low-level actions solely based on 2D visual observations. Moreover, we design a Hybrid Training Paradigm (HTP) that consists of three stages, i.e., Action-/TBA-SFT, Nav-GPRO, and Online RL stages. Each stage contains specifically designed learning policies and rewards. Importantly, for TBA-SFT and Nav-GRPO designs, we are inspired by the OpenAI-o1 and DeepSeek-R1, which show impressive reasoning ability via thinking-before-answer. Thus, we aim to investigate how to achieve thinking-before-action in the embodied navigation field, to improve model's reasoning ability toward generalists. Specifically, we propose TBA-SFT to utilize the TBA-CoT dataset to fine-tune the model as a cold-start phrase and then leverage Nav-GPRO to improve its thinking ability. Finally, OctoNav-R1 shows superior performance compared with previous methods.

関連論文

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開
  3. 3newmoとKDDIが資本業務提携 2028年に堺市で自動運転タクシー商用化目指す提携・買収
  4. 4NVIDIA、CoRLでGR00T 2とIsaac Lab 3.0を発表新製品・発表
  5. 5UC BerkeleyとNVIDIAの研究者、AIエージェントでロボット制御コードを仮想環境で改善研究・技術

PR・アフィリエイト広告

OZ GAMING(GPU搭載BTOゲーミングPC)

OZ GAMING(GPU搭載BTOゲーミングPC)の公式広告バナーOZ GAMINGで完成品PCを見る ↗
PR本紙発行元 EmplifAI