スケールフィードバックによる報酬関数の学習
Learning Reward Functions from Scale Feedback
ユーザーがスライダーで好みの度合いを伝える「スケールフィードバック」を提案し、その確率モデルに基づくロボットの報酬学習手法を開発した。シミュレーションと2件のユーザー研究で有効性を示した。
著者: Nils Wilde, Erdem Bıyık, Dorsa Sadigh, Stephen L. Smith
分類: cs.RO, cs.AI, cs.LG
原文アブストラクト
Today's robots are increasingly interacting with people and need to efficiently learn inexperienced user's preferences. A common framework is to iteratively query the user about which of two presented robot trajectories they prefer. While this minimizes the users effort, a strict choice does not yield any information on how much one trajectory is preferred. We propose scale feedback, where the user utilizes a slider to give more nuanced information. We introduce a probabilistic model on how users would provide feedback and derive a learning framework for the robot. We demonstrate the performance benefit of slider feedback in simulations, and validate our approach in two user studies suggesting that scale feedback enables more effective learning in practice.
