一人称視点動画から活動文脈の事前知識を抽出しエージェント行動を形成する
Shaping embodied agent behavior with activity-context priors from egocentric video
人間の一人称視点動画から物体同士の共起関係(活動文脈の事前知識)を学習し、補助報酬としてロボットエージェントに与えることで、複雑な家庭内タスクの学習を大幅に加速させる手法を提案した。
著者: Tushar Nagarajan, Kristen Grauman
分類: cs.CV, cs.RO
原文アブストラクト
Complex physical tasks entail a sequence of object interactions, each with its own preconditions -- which can be difficult for robotic agents to learn efficiently solely through their own experience. We introduce an approach to discover activity-context priors from in-the-wild egocentric video captured with human worn cameras. For a given object, an activity-context prior represents the set of other compatible objects that are required for activities to succeed (e.g., a knife and cutting board brought together with a tomato are conducive to cutting). We encode our video-based prior as an auxiliary reward function that encourages an agent to bring compatible objects together before attempting an interaction. In this way, our model translates everyday human experience into embodied agent skills. We demonstrate our idea using egocentric EPIC-Kitchens video of people performing unscripted kitchen activities to benefit virtual household robotic agents performing various complex tasks in AI2-iTHOR, significantly accelerating agent learning. Project page: http://vision.cs.utexas.edu/projects/ego-rewards/
