LiDAR点群の自己教師あり学習のためのマルチスケール近傍占有マスク付きオートエンコーダ
Multi-Scale Neighborhood Occupancy Masked Autoencoder for Self-Supervised Learning in LiDAR Point Clouds
LiDAR点群の自己教師あり学習において、非マスクボクセルの近傍のみで占有再構成を行うことで情報漏洩と計算コストを抑え、複数スケールのマスク生成で多様な物体特徴を捉えるNOMAEを提案し、nuScenesとWaymoで最先端性能を達成した。
著者: Mohamed Abdelsamad, Michael Ulrich, Claudius Gläser, Abhinav Valada
分類: cs.CV, cs.AI, cs.RO
原文アブストラクト
Masked autoencoders (MAE) have shown tremendous potential for self-supervised learning (SSL) in vision and beyond. However, point clouds from LiDARs used in automated driving are particularly challenging for MAEs since large areas of the 3D volume are empty. Consequently, existing work suffers from leaking occupancy information into the decoder and has significant computational complexity, thereby limiting the SSL pre-training to only 2D bird's eye view encoders in practice. In this work, we propose the novel neighborhood occupancy MAE (NOMAE) that overcomes the aforementioned challenges by employing masked occupancy reconstruction only in the neighborhood of non-masked voxels. We incorporate voxel masking and occupancy reconstruction at multiple scales with our proposed hierarchical mask generation technique to capture features of objects of different sizes in the point cloud. NOMAEs are extremely flexible and can be directly employed for SSL in existing 3D architectures. We perform extensive evaluations on the nuScenes and Waymo Open datasets for the downstream perception tasks of semantic segmentation and 3D object detection, comparing with both discriminative and generative SSL methods. The results demonstrate that NOMAE sets the new state-of-the-art on multiple benchmarks for multiple point cloud perception tasks.
