π2vec: 後継特徴量による方策表現
$\pi2\text{vec}$: Policy Representations with Successor Features
黒箱方策の振る舞いを、基盤モデル特徴量の統計変化として捉えた特徴ベクトルで表現する手法を提案し、オフライン方策選択に応用する。
著者: Gianluca Scarpellini, Ksenia Konyushkova, Claudio Fantacci, Tom Le Paine, Yutian Chen, Misha Denil
分類: cs.LG, cs.RO
原文アブストラクト
This paper describes $\pi2\text{vec}$, a method for representing behaviors of black box policies as feature vectors. The policy representations capture how the statistics of foundation model features change in response to the policy behavior in a task agnostic way, and can be trained from offline data, allowing them to be used in offline policy selection. This work provides a key piece of a recipe for fusing together three modern lines of research: Offline policy evaluation as a counterpart to offline RL, foundation models as generic and powerful state representations, and efficient policy selection in resource constrained environments.
