多目的方策最適化の分布的アプローチ
A Distributional View on Multi-Objective Policy Optimization
複数の競合する目的をスケール不変な選好で扱える多目的強化学習アルゴリズムを提案し、目的ごとの行動分布を学習してパラメトリック方策に統合、高次元ロボティクスタスクで有効性を示した。
著者: Abbas Abdolmaleki, Sandy H. Huang, Leonard Hasenclever, Michael Neunert, H. Francis Song, Martina Zambelli, Murilo F. Martins, Nicolas Heess, Raia Hadsell, Martin Riedmiller
分類: cs.LG, cs.AI, cs.RO, stat.ML
原文アブストラクト
Many real-world problems require trading off multiple competing objectives. However, these objectives are often in different units and/or scales, which can make it challenging for practitioners to express numerical preferences over objectives in their native units. In this paper we propose a novel algorithm for multi-objective reinforcement learning that enables setting desired preferences for objectives in a scale-invariant way. We propose to learn an action distribution for each objective, and we use supervised learning to fit a parametric policy to a combination of these distributions. We demonstrate the effectiveness of our approach on challenging high-dimensional real and simulated robotics tasks, and show that setting different preferences in our framework allows us to trace out the space of nondominated solutions.
