自動運転における3D物体検出のオフライン評価について
On Offline Evaluation of 3D Object Detection for Autonomous Driving
自動運転スタックに16の物体検出モデルを統合し、オフライン検出指標が実際の運転性能をどれだけ予測できるかをCARLAシミュレータで検証した。
著者: Tim Schreier, Katrin Renz, Andreas Geiger, Kashyap Chitta
分類: cs.CV, cs.RO
原文アブストラクト
Prior work in 3D object detection evaluates models using offline metrics like average precision since closed-loop online evaluation on the downstream driving task is costly. However, it is unclear how indicative offline results are of driving performance. In this work, we perform the first empirical evaluation measuring how predictive different detection metrics are of driving performance when detectors are integrated into a full self-driving stack. We conduct extensive experiments on urban driving in the CARLA simulator using 16 object detection models. We find that the nuScenes Detection Score has a higher correlation to driving performance than the widely used average precision metric. In addition, our results call for caution on the exclusive reliance on the emerging class of `planner-centric' metrics.
