潜在条件付き方策勾配による多目的深層強化学習
Latent-Conditioned Policy Gradient for Multi-Objective Deep Reinforcement Learning
単一のニューラルネットワークを方策勾配で訓練し、線形スカラー化に頼らずにパレート最適方策群を一度の学習で近似的に獲得する多目的強化学習手法を提案した。
著者: Takuya Kanazawa, Chetan Gupta
分類: cs.LG, cs.RO
原文アブストラクト
Sequential decision making in the real world often requires finding a good balance of conflicting objectives. In general, there exist a plethora of Pareto-optimal policies that embody different patterns of compromises between objectives, and it is technically challenging to obtain them exhaustively using deep neural networks. In this work, we propose a novel multi-objective reinforcement learning (MORL) algorithm that trains a single neural network via policy gradient to approximately obtain the entire Pareto set in a single run of training, without relying on linear scalarization of objectives. The proposed method works in both continuous and discrete action spaces with no design change of the policy network. Numerical experiments in benchmark environments demonstrate the practicality and efficacy of our approach in comparison to standard MORL baselines.
