デモンストレーションから記憶メカニズムを学習する意思決定
Learning Memory Mechanisms for Decision Making through Demonstrations
専門家の記憶依存関係をペアで示すことで、Transformerの注意機構を調整し、部分観測環境下での長期記憶を要する意思決定を改善する手法を提案。
著者: William Yue, Bo Liu, Peter Stone
分類: cs.LG, cs.RO
原文アブストラクト
In Partially Observable Markov Decision Processes, integrating an agent's history into memory poses a significant challenge for decision-making. Traditional imitation learning, relying on observation-action pairs for expert demonstrations, fails to capture the expert's memory mechanisms used in decision-making. To capture memory processes as demonstrations, we introduce the concept of memory dependency pairs $(p, q)$ indicating that events at time $p$ are recalled for decision-making at time $q$. We introduce AttentionTuner to leverage memory dependency pairs in Transformers and find significant improvements across several tasks compared to standard Transformers when evaluated on Memory Gym and the Long-term Memory Benchmark. Code is available at https://github.com/WilliamYue37/AttentionTuner.
