深層姿勢回帰ネットワークからの知識蒸留による効率的な視覚オドメトリ
Distilling Knowledge From a Deep Pose Regressor Network
姿勢回帰ネットワークの知識を蒸留する際、教師の信頼度を損失で測り注意機構付き模倣・ヒント学習で生徒に転移する手法を提案。パラメータを最大92.95%削減し計算を2.12倍高速化した。
著者: Muhamad Risqi U. Saputra, Pedro P. B. de Gusmao, Yasin Almalioglu, Andrew Markham, Niki Trigoni
分類: cs.CV, cs.LG, cs.RO
原文アブストラクト
This paper presents a novel method to distill knowledge from a deep pose regressor network for efficient Visual Odometry (VO). Standard distillation relies on "dark knowledge" for successful knowledge transfer. As this knowledge is not available in pose regression and the teacher prediction is not always accurate, we propose to emphasize the knowledge transfer only when we trust the teacher. We achieve this by using teacher loss as a confidence score which places variable relative importance on the teacher prediction. We inject this confidence score to the main training task via Attentive Imitation Loss (AIL) and when learning the intermediate representation of the teacher through Attentive Hint Training (AHT) approach. To the best of our knowledge, this is the first work which successfully distill the knowledge from a deep pose regression network. Our evaluation on the KITTI and Malaga dataset shows that we can keep the student prediction close to the teacher with up to 92.95% parameter reduction and 2.12x faster in computation time.
