変分推論arXiv:2001.10025
多変量ガウス分布の自然勾配降下法による変分推論
Multivariate Gaussian Variational Inference by Natural Gradient Descent
多変量ガウス分布の変分推論において、平均と逆共分散行列をパラメータ化する自然勾配降下法を導出し、対称性を考慮した簡潔な更新式を提案した。
分類: stat.ML, cs.LG, cs.RO, math.ST, stat.TH
原文アブストラクト
This short note reviews so-called Natural Gradient Descent (NGD) for multivariate Gaussians. The Fisher Information Matrix (FIM) is derived for several different parameterizations of Gaussians. Careful attention is paid to the symmetric nature of the covariance matrix when calculating derivatives. We show that there are some advantages to choosing a parameterization comprising the mean and inverse covariance matrix and provide a simple NGD update that accounts for the symmetric (and sparse) nature of the inverse covariance matrix.
