日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

マニピュレーションarXiv:1601.02705

Robobarista:深層マルチモーダル埋め込みによる新規物体の操作学習

Robobarista: Learning to Manipulate Novel Objects via Deep Multimodal Embedding

シェア:XThreadsFacebookLINEはてブBluesky

点群・自然言語・操作軌道を共有埋め込み空間に写像し、類似した操作部位を持つ未知の家電にも操作戦略を転移できる操作計画手法を提案。クラウドソーシングで収集した116物体・249部位の実演データで検証し、未見の器具でラテを作れることを示した。

著者: Jaeyong Sung, Seok Hyun Jin, Ian Lenz, Ashutosh Saxena

分類: cs.RO, cs.AI, cs.LG

原文アブストラクト

There is a large variety of objects and appliances in human environments, such as stoves, coffee dispensers, juice extractors, and so on. It is challenging for a roboticist to program a robot for each of these object types and for each of their instantiations. In this work, we present a novel approach to manipulation planning based on the idea that many household objects share similarly-operated object parts. We formulate the manipulation planning as a structured prediction problem and learn to transfer manipulation strategy across different objects by embedding point-cloud, natural language, and manipulation trajectory data into a shared embedding space using a deep neural network. In order to learn semantically meaningful spaces throughout our network, we introduce a method for pre-training its lower layers for multimodal feature embedding and a method for fine-tuning this embedding space using a loss-based margin. In order to collect a large number of manipulation demonstrations for different objects, we develop a new crowd-sourcing platform called Robobarista. We test our model on our dataset consisting of 116 objects and appliances with 249 parts along with 250 language instructions, for which there are 1225 crowd-sourced manipulation demonstrations. We further show that our robot with our model can even prepare a cup of a latte with appliances it has never seen before.

関連論文

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2newmoとKDDIが資本業務提携 2028年に堺市で自動運転タクシー商用化目指す提携・買収
  3. 3GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開
  4. 4NVIDIA、CoRLでGR00T 2とIsaac Lab 3.0を発表新製品・発表
  5. 5UC BerkeleyとNVIDIAの研究者、AIエージェントでロボット制御コードを仮想環境で改善研究・技術

PR・アフィリエイト広告

OZ GAMING(GPU搭載BTOゲーミングPC)

OZ GAMING(GPU搭載BTOゲーミングPC)の公式広告バナーOZ GAMINGで完成品PCを見る ↗
PR本紙発行元 EmplifAI