現場で学ぶ：人とロボットの相互作用における長期的な行動適応
Learning on the Job: Long-Term Behavioural Adaptation in Human-Robot Interactions
公共空間に長期配備された案内ロボットが、強化学習（UCBVI）とエンゲージメント報酬モデルを用いてユーザーとの対話からオンラインで行動を適応させ、ツアー完遂率を30%向上させた研究。
著者: Francesco Del Duchetto, Marc Hanheide
分類: cs.RO, cs.HC, cs.LG
原文アブストラクト
In this work, we propose a framework for allowing autonomous robots deployed for extended periods of time in public spaces to adapt their own behaviour online from user interactions. The robot behaviour planning is embedded in a Reinforcement Learning (RL) framework, where the objective is maximising the level of overall user engagement during the interactions. We use the Upper-Confidence-Bound Value-Iteration (UCBVI) algorithm, which gives a helpful way of managing the exploration-exploitation trade-off for real-time interactions. An engagement model trained end-to-end generates the reward function in real-time during policy execution. We test this approach in a public museum in Lincoln (UK), where the robot is deployed as a tour guide for the visitors. Results show that after a couple of months of exploration, the robot policy learned to maintain the engagement of users for longer, with an increase of 22.8% over the initial static policy in the number of items visited during the tour and a 30% increase in the probability of completing the tour. This work is a promising step toward behavioural adaptation in long-term scenarios for robotics applications in social settings.
