日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

報酬学習arXiv:2006.13208

特徴拡張型報酬学習:人手入力の再考

Feature Expansive Reward Learning: Rethinking Human Input

シェア:XThreadsFacebookLINEはてブBluesky

ロボットの報酬関数に不足する特徴を、人間がその特徴が強く現れる状態から現れない状態へ誘導する新たな入力で明示的に教え、生状態空間から特徴を学習して報酬に統合する手法を提案。深層逆強化学習より少ないサンプルで汎化性能が向上することを実機7自由度マニピュレータとシミュレーションのユーザー研究で示した。

著者: Andreea Bobu, Marius Wiggert, Claire Tomlin, Anca D. Dragan

分類: cs.RO, cs.AI, cs.HC, cs.LG, stat.ML

原文アブストラクト

When a person is not satisfied with how a robot performs a task, they can intervene to correct it. Reward learning methods enable the robot to adapt its reward function online based on such human input, but they rely on handcrafted features. When the correction cannot be explained by these features, recent work in deep Inverse Reinforcement Learning (IRL) suggests that the robot could ask for task demonstrations and recover a reward defined over the raw state space. Our insight is that rather than implicitly learning about the missing feature(s) from demonstrations, the robot should instead ask for data that explicitly teaches it about what it is missing. We introduce a new type of human input in which the person guides the robot from states where the feature being taught is highly expressed to states where it is not. We propose an algorithm for learning the feature from the raw state space and integrating it into the reward function. By focusing the human input on the missing feature, our method decreases sample complexity and improves generalization of the learned reward over the above deep IRL baseline. We show this in experiments with a physical 7DOF robot manipulator, as well as in a user study conducted in a simulated environment.

関連論文

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2newmoとKDDIが資本業務提携 2028年に堺市で自動運転タクシー商用化目指す提携・買収
  3. 3GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開
  4. 4NVIDIA、CoRLでGR00T 2とIsaac Lab 3.0を発表新製品・発表
  5. 5UC BerkeleyとNVIDIAの研究者、AIエージェントでロボット制御コードを仮想環境で改善研究・技術

PR・アフィリエイト広告

OZ GAMING(GPU搭載BTOゲーミングPC)

OZ GAMING(GPU搭載BTOゲーミングPC)の公式広告バナーOZ GAMINGで完成品PCを見る ↗
PR本紙発行元 EmplifAI