不変因果特徴を持つ世界モデルの対照的教師なし学習
Contrastive Unsupervised Learning of World Model with Invariant Causal Features
対照学習と深度予測による補助タスクで不変な因果特徴を獲得する世界モデルを提案し、分布外ナビゲーションやsim-to-real転移で高性能を示した。
著者: Rudra P. K. Poudel, Harit Pandya, Roberto Cipolla
分類: cs.LG, cs.AI, cs.CV, cs.RO, stat.ML
原文アブストラクト
In this paper we present a world model, which learns causal features using the invariance principle. In particular, we use contrastive unsupervised learning to learn the invariant causal features, which enforces invariance across augmentations of irrelevant parts or styles of the observation. The world-model-based reinforcement learning methods independently optimize representation learning and the policy. Thus naive contrastive loss implementation collapses due to a lack of supervisory signals to the representation learning module. We propose an intervention invariant auxiliary task to mitigate this issue. Specifically, we utilize depth prediction to explicitly enforce the invariance and use data augmentation as style intervention on the RGB observation space. Our design leverages unsupervised representation learning to learn the world model with invariant causal features. Our proposed method significantly outperforms current state-of-the-art model-based and model-free reinforcement learning methods on out-of-distribution point navigation tasks on the iGibson dataset. Moreover, our proposed model excels at the sim-to-real transfer of our perception learning module. Finally, we evaluate our approach on the DeepMind control suite and enforce invariance only implicitly since depth is not available. Nevertheless, our proposed model performs on par with the state-of-the-art counterpart.
