PDiT: 深層強化学習のための知覚と意思決定のトランスフォーマーを交互に配置する手法
PDiT: Interleaving Perception and Decision-making Transformers for Deep Reinforcement Learning
知覚用と意思決定用の2つのTransformerを交互に配置したPDiTネットワークを提案し、画像・固有感覚・画像言語の観測下でのオンライン・オフライン強化学習において高性能と解釈可能な特徴表現を実現した。
著者: Hangyu Mao, Rui Zhao, Ziyue Li, Zhiwei Xu, Hao Chen, Yiqun Chen, Bin Zhang, Zhen Xiao, Junge Zhang, Jiangjin Yin
分類: cs.LG, cs.AI, cs.RO, cs.SY, eess.SY
原文アブストラクト
Designing better deep networks and better reinforcement learning (RL) algorithms are both important for deep RL. This work studies the former. Specifically, the Perception and Decision-making Interleaving Transformer (PDiT) network is proposed, which cascades two Transformers in a very natural way: the perceiving one focuses on \emph{the environmental perception} by processing the observation at the patch level, whereas the deciding one pays attention to \emph{the decision-making} by conditioning on the history of the desired returns, the perceiver's outputs, and the actions. Such a network design is generally applicable to a lot of deep RL settings, e.g., both the online and offline RL algorithms under environments with either image observations, proprioception observations, or hybrid image-language observations. Extensive experiments show that PDiT can not only achieve superior performance than strong baselines in different settings but also extract explainable feature representations. Our code is available at \url{https://github.com/maohangyu/PDiT}.
