日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

VLAarXiv:2109.02823

音声指示で動くロボットのための視聴覚表現学習

Learning Visual-Audio Representations for Voice-Controlled Robots

シェア:XThreadsFacebookLINEはてブBluesky

画像と音声コマンドを結びつける視聴覚表現を学習し、その表現から生成した報酬で強化学習を行うことで、ラベルを大幅に削減しつつ音声指示で動作するロボットを実現した研究。

著者: Peixin Chang, Shuijing Liu, D. Livingston McPherson, Katherine Driggs-Campbell

分類: cs.RO, cs.AI

原文アブストラクト

Based on the recent advancements in representation learning, we propose a novel pipeline for task-oriented voice-controlled robots with raw sensor inputs. Previous methods rely on a large number of labels and task-specific reward functions. Not only can such an approach hardly be improved after the deployment, but also has limited generalization across robotic platforms and tasks. To address these problems, our pipeline first learns a visual-audio representation (VAR) that associates images and sound commands. Then the robot learns to fulfill the sound command via reinforcement learning using the reward generated by the VAR. We demonstrate our approach with various sound types, robots, and tasks. We show that our method outperforms previous work with much fewer labels. We show in both the simulated and real-world experiments that the system can self-improve in previously unseen scenarios given a reasonable number of newly labeled data.

関連論文

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2newmoとKDDIが資本業務提携 2028年に堺市で自動運転タクシー商用化目指す提携・買収
  3. 3GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開
  4. 4NVIDIA、CoRLでGR00T 2とIsaac Lab 3.0を発表新製品・発表
  5. 5UC BerkeleyとNVIDIAの研究者、AIエージェントでロボット制御コードを仮想環境で改善研究・技術

PR・アフィリエイト広告

MINISFORUM(ミニPC)

MINISFORUM(ミニPC)の公式広告バナーMINISFORUM公式ストアを見る ↗
PR本紙発行元 EmplifAI