オンライン選好からの能動的報酬学習
Active Reward Learning from Online Preferences
人間への負担を抑えつつ、オンラインで提示する一対比較の選好クエリを設計・選択し、ロボット方策を効率的に適応させる手法を提案した。
著者: Vivek Myers, Erdem Bıyık, Dorsa Sadigh
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
Robot policies need to adapt to human preferences and/or new environments. Human experts may have the domain knowledge required to help robots achieve this adaptation. However, existing works often require costly offline re-training on human feedback, and those feedback usually need to be frequent and too complex for the humans to reliably provide. To avoid placing undue burden on human experts and allow quick adaptation in critical real-world situations, we propose designing and sparingly presenting easy-to-answer pairwise action preference queries in an online fashion. Our approach designs queries and determines when to present them to maximize the expected value derived from the queries' information. We demonstrate our approach with experiments in simulation, human user studies, and real robot experiments. In these settings, our approach outperforms baseline techniques while presenting fewer queries to human experts. Experiment videos, code and appendices are found at https://sites.google.com/view/onlineactivepreferences.
