EnQuery: ロボットナビゲーションの選好整合に向けた多様なクエリ生成のためのアンサンブル方策
EnQuery: Ensemble Policies for Diverse Query-Generation in Preference Alignment of Robot Navigation
人間のフィードバックによる強化学習でナビゲーション方策をユーザ選好に合わせるため、正則化項で行動多様性を確保したアンサンブル方策により、少ないクエリで多様な経路候補を生成する手法を提案した。
著者: Jorge de Heuvel, Florian Seiler, Maren Bennewitz
分類: cs.RO
原文アブストラクト
To align mobile robot navigation policies with user preferences through reinforcement learning from human feedback (RLHF), reliable and behavior-diverse user queries are required. However, deterministic policies fail to generate a variety of navigation trajectory suggestions for a given navigation task. In this paper, we introduce EnQuery, a query generation approach using an ensemble of policies that achieve behavioral diversity through a regularization term. For a given navigation task, EnQuery produces multiple navigation trajectory suggestions, thereby optimizing the efficiency of preference data collection with fewer queries. Our methodology demonstrates superior performance in aligning navigation policies with user preferences in low-query regimes, offering enhanced policy convergence from sparse preference queries. The evaluation is complemented with a novel explainability representation, capturing full scene navigation behavior of the mobile robot in a single plot. Our code is available online at https://github.com/hrl-bonn/EnQuery.
