深層回帰における不確かさ定量化の評価指標
Uncertainty Quantification Metrics for Deep Regression
深層回帰モデルの予測不確かさを評価する指標（AUSE、キャリブレーション誤差、スピアマン順位相関、NLL）を合成データで比較し、キャリブレーション誤差が最も安定で解釈しやすいことなどを明らかにした。
著者: Simon Kristoffersson Lind, Ziliang Xiong, Per-Erik Forssén, Volker Krüger
分類: cs.LG, cs.RO
原文アブストラクト
When deploying deep neural networks on robots or other physical systems, the learned model should reliably quantify predictive uncertainty. A reliable uncertainty allows downstream modules to reason about the safety of its actions. In this work, we address metrics for evaluating such an uncertainty. Specifically, we focus on regression tasks, and investigate Area Under Sparsification Error (AUSE), Calibration Error, Spearman's Rank Correlation, and Negative Log-Likelihood (NLL). Using synthetic regression datasets, we look into how those metrics behave under four typical types of uncertainty, their stability regarding the size of the test set, and reveal their strengths and weaknesses. Our results indicate that Calibration Error is the most stable and interpretable metric, but AUSE and NLL also have their respective use cases. We discourage the usage of Spearman's Rank Correlation for evaluating uncertainties and recommend replacing it with AUSE.
