強化学習arXiv:2407.12163
ベルマン拡散モデル
Bellman Diffusion Models
拡散モデルを方策の後続状態測度のモデルとして用い、ベルマン流束制約を課すことで拡散ステップ分布の簡単なベルマン更新が導かれることを示した。
著者: Liam Schramm, Abdeslam Boularias
分類: cs.LG, cs.RO
原文アブストラクト
Diffusion models have seen tremendous success as generative architectures. Recently, they have been shown to be effective at modelling policies for offline reinforcement learning and imitation learning. We explore using diffusion as a model class for the successor state measure (SSM) of a policy. We find that enforcing the Bellman flow constraints leads to a simple Bellman update on the diffusion step distribution.
