教師なし特徴選択による透明な報酬モデルの学習
Learning Transparent Reward Models via Unsupervised Feature Selection
専門家のデモデータから、自動選択した状態特徴を用いて明示的な形のコンパクトな報酬関数を推定し、強化学習で専門家に近い方策を学習する手法を提案した論文。
著者: Daulet Baimukashev, Gokhan Alcan, Kevin Sebastian Luck, Ville Kyrki
分類: cs.RO
原文アブストラクト
In complex real-world tasks such as robotic manipulation and autonomous driving, collecting expert demonstrations is often more straightforward than specifying precise learning objectives and task descriptions. Learning from expert data can be achieved through behavioral cloning or by learning a reward function, i.e., inverse reinforcement learning. The latter allows for training with additional data outside the training distribution, guided by the inferred reward function. We propose a novel approach to construct compact and transparent reward models from automatically selected state features. These inferred rewards have an explicit form and enable the learning of policies that closely match expert behavior by training standard reinforcement learning algorithms from scratch. We validate our method's performance in various robotic environments with continuous and high-dimensional state spaces. Webpage: \url{https://sites.google.com/view/transparent-reward}.
