スパース情報形式によるニューラルネットワークのモデル不確実性推定
Estimating Model Uncertainty of Neural Networks in Sparse Information Form
情報行列のスペクトルが疎になりやすい性質を利用し、低ランク近似と新たなスパース化アルゴリズムでDNNのモデル不確実性を効率的に表現・サンプリングする手法を提案した。
著者: Jongseok Lee, Matthias Humt, Jianxiang Feng, Rudolph Triebel
分類: cs.LG, cs.CV, cs.RO, stat.ML
原文アブストラクト
We present a sparse representation of model uncertainty for Deep Neural Networks (DNNs) where the parameter posterior is approximated with an inverse formulation of the Multivariate Normal Distribution (MND), also known as the information form. The key insight of our work is that the information matrix, i.e. the inverse of the covariance matrix tends to be sparse in its spectrum. Therefore, dimensionality reduction techniques such as low rank approximations (LRA) can be effectively exploited. To achieve this, we develop a novel sparsification algorithm and derive a cost-effective analytical sampler. As a result, we show that the information form can be scalably applied to represent model uncertainty in DNNs. Our exhaustive theoretical analysis and empirical evaluations on various benchmarks show the competitiveness of our approach over the current methods.
