転移可能な報酬分解による逆制約学習と汎化
Inverse Constraint Learning and Generalization by Transferable Reward Decomposition
デモンストレーションから制約を推定する逆制約学習において、タスク指向報酬とタスク非依存制約を同時に推論するTCLを提案し、新規環境での汎化性能を向上させた。
著者: Jaehwi Jang, Minjae Song, Daehyung Park
分類: cs.RO
原文アブストラクト
We present the problem of inverse constraint learning (ICL), which recovers constraints from demonstrations to autonomously reproduce constrained skills in new scenarios. However, ICL suffers from an ill-posed nature, leading to inaccurate inference of constraints from demonstrations. To figure it out, we introduce a transferable constraint learning (TCL) algorithm that jointly infers a task-oriented reward and a task-agnostic constraint, enabling the generalization of learned skills. Our method TCL additively decomposes the overall reward into a task reward and its residual as soft constraints, maximizing policy divergence between task- and constraint-oriented policies to obtain a transferable constraint. Evaluating our method and five baselines in three simulated environments, we show TCL outperforms state-of-the-art IRL and ICL algorithms, achieving up to a $72\%$ higher task-success rates with accurate decomposition compared to the next best approach in novel scenarios. Further, we demonstrate the robustness of TCL on two real-world robotic tasks.
