人間の動画から抽出するロボットのためのアフォーダンス表現
Affordances from Human Videos as a Versatile Representation for Robotics
インターネット上の人間の行動動画から視覚的アフォーダンスモデルを学習し、ロボットが複雑なタスクを実行できる汎用的な表現として活用する手法を提案。
著者: Shikhar Bahl, Russell Mendonca, Lili Chen, Unnat Jain, Deepak Pathak
分類: cs.RO, cs.AI, cs.CV, cs.LG, cs.NE
原文アブストラクト
Building a robot that can understand and learn to interact by watching humans has inspired several vision problems. However, despite some successful results on static datasets, it remains unclear how current models can be used on a robot directly. In this paper, we aim to bridge this gap by leveraging videos of human interactions in an environment centric manner. Utilizing internet videos of human behavior, we train a visual affordance model that estimates where and how in the scene a human is likely to interact. The structure of these behavioral affordances directly enables the robot to perform many complex tasks. We show how to seamlessly integrate our affordance model with four robot learning paradigms including offline imitation learning, exploration, goal-conditioned learning, and action parameterization for reinforcement learning. We show the efficacy of our approach, which we call VRB, across 4 real world environments, over 10 different tasks, and 2 robotic platforms operating in the wild. Results, visualizations and videos at https://robo-affordances.github.io/
