因果的好奇心:因果表現学習のための自己教師あり実験を発見する強化学習エージェント
Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learning
強化学習エージェントに「因果的好奇心」という新たな内的報酬を与え、環境内で自ら実験を行うことで因果的要因を階層的に推論・表現学習させる手法を提案した論文。
著者: Sumedh A. Sontakke, Arash Mehrjou, Laurent Itti, Bernhard Schölkopf
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
Animals exhibit an innate ability to learn regularities of the world through interaction. By performing experiments in their environment, they are able to discern the causal factors of variation and infer how they affect the world's dynamics. Inspired by this, we attempt to equip reinforcement learning agents with the ability to perform experiments that facilitate a categorization of the rolled-out trajectories, and to subsequently infer the causal factors of the environment in a hierarchical manner. We introduce {\em causal curiosity}, a novel intrinsic reward, and show that it allows our agents to learn optimal sequences of actions and discover causal factors in the dynamics of the environment. The learned behavior allows the agents to infer a binary quantized representation for the ground-truth causal factors in every environment. Additionally, we find that these experimental behaviors are semantically meaningful (e.g., our agents learn to lift blocks to categorize them by weight), and are learnt in a self-supervised manner with approximately 2.5 times less data than conventional supervised planners. We show that these behaviors can be re-purposed and fine-tuned (e.g., from lifting to pushing or other downstream tasks). Finally, we show that the knowledge of causal factor representations aids zero-shot learning for more complex tasks. Visit https://sites.google.com/usc.edu/causal-curiosity/home for website.
