LiFT: 基盤モデルを教師とする教師なし強化学習
LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teachers
大規模言語モデルが環境に基づくタスク指示を生成し、視覚言語モデルが報酬を与えることで、人間のフィードバックなしに意味のあるスキルを学習する枠組みを提案。MineDojo環境で有効性を示す。
著者: Taewook Nam, Juyong Lee, Jesse Zhang, Sung Ju Hwang, Joseph J. Lim, Karl Pertsch
分類: cs.LG, cs.AI, cs.RO
原文アブストラクト
We propose a framework that leverages foundation models as teachers, guiding a reinforcement learning agent to acquire semantically meaningful behavior without human feedback. In our framework, the agent receives task instructions grounded in a training environment from large language models. Then, a vision-language model guides the agent in learning the multi-task language-conditioned policy by providing reward feedback. We demonstrate that our method can learn semantically meaningful skills in a challenging open-ended MineDojo environment while prior unsupervised skill discovery methods struggle. Additionally, we discuss observed challenges of using off-the-shelf foundation models as teachers and our efforts to address them.
