物理的ヒューマンフィードバックからの学習：物体中心のワンショット適応手法
Learning from Physical Human Feedback: An Object-Centric One-Shot Adaptation Method
人間の介入を物体ごとの選好として解釈し、エージェントと物体間の一般的な相互作用は固定したまま物体固有の選好のみをオンライン更新することで、1回の介入だけで新たな行動を獲得する適応手法を提案した。
著者: Alvin Shek, Bo Ying Su, Rui Chen, Changliu Liu
分類: cs.RO, cs.AI, cs.HC, cs.LG
原文アブストラクト
For robots to be effectively deployed in novel environments and tasks, they must be able to understand the feedback expressed by humans during intervention. This can either correct undesirable behavior or indicate additional preferences. Existing methods either require repeated episodes of interactions or assume prior known reward features, which is data-inefficient and can hardly transfer to new tasks. We relax these assumptions by describing human tasks in terms of object-centric sub-tasks and interpreting physical interventions in relation to specific objects. Our method, Object Preference Adaptation (OPA), is composed of two key stages: 1) pre-training a base policy to produce a wide variety of behaviors, and 2) online-updating according to human feedback. The key to our fast, yet simple adaptation is that general interaction dynamics between agents and objects are fixed, and only object-specific preferences are updated. Our adaptation occurs online, requires only one human intervention (one-shot), and produces new behaviors never seen during training. Trained on cheap synthetic data instead of expensive human demonstrations, our policy correctly adapts to human perturbations on realistic tasks on a physical 7DOF robot. Videos, code, and supplementary material are provided.
