文脈依存の辞書式報酬選好に基づく多目的計画
Multi-Objective Planning with Contextual Lexicographic Reward Preferences
文脈ごとに目的の優先順位が変わる状況で、専門家の軌跡から状態と文脈の対応をベイズ推定し、各順序の政策を統合して巡回のない政策を計画する枠組みを提案した。
著者: Pulkit Rustagi, Yashwanthi Anand, Sandhya Saisubramanian
分類: cs.AI, cs.RO, cs.SY, eess.SY
原文アブストラクト
Autonomous agents are often required to plan under multiple objectives whose preference ordering varies based on context. The agent may encounter multiple contexts during its course of operation, each imposing a distinct lexicographic ordering over the objectives, with potentially different reward functions associated with each context. Existing approaches to multi-objective planning typically consider a single preference ordering over the objectives, across the state space, and do not support planning under multiple objective orderings within an environment. We present Contextual Lexicographic Markov Decision Process (CLMDP), a framework that enables planning under varying lexicographic objective orderings, depending on the context. In a CLMDP, both the objective ordering at a state and the associated reward functions are determined by the context. We employ a Bayesian approach to infer a state-context mapping from expert trajectories. Our algorithm to solve a CLMDP first computes a policy for each objective ordering and then combines them into a single context-aware policy that is valid and cycle-free. The effectiveness of the proposed approach is evaluated in simulation and using a mobile robot.
