敵対的環境生成による分布ロバスト方策学習
Distributionally Robust Policy Learning via Adversarial Environment Generation
環境の生成モデルを学習し、潜在空間での勾配上昇により現実的な敵対的環境を生成することで、分布シフトに対してロバストな制御方策を学習する手法DRAGENを提案した。
著者: Allen Z. Ren, Anirudha Majumdar
分類: cs.RO, cs.LG
原文アブストラクト
Our goal is to train control policies that generalize well to unseen environments. Inspired by the Distributionally Robust Optimization (DRO) framework, we propose DRAGEN - Distributionally Robust policy learning via Adversarial Generation of ENvironments - for iteratively improving robustness of policies to realistic distribution shifts by generating adversarial environments. The key idea is to learn a generative model for environments whose latent variables capture cost-predictive and realistic variations in environments. We perform DRO with respect to a Wasserstein ball around the empirical distribution of environments by generating realistic adversarial environments via gradient ascent on the latent space. We demonstrate strong Out-of-Distribution (OoD) generalization in simulation for (i) swinging up a pendulum with onboard vision and (ii) grasping realistic 3D objects. Grasping experiments on hardware demonstrate better sim2real performance compared to domain randomization.
