言語支援による3Dシーン理解
Language-Assisted 3D Scene Understanding
LLMでテキストを拡張し、統計的・訓練不要な特徴選択で冗長性を除去することで、点群特徴学習を言語支援する手法を提案。3Dセマンティックセグメンテーション等で高性能を達成。
著者: Yanmin Wu, Qiankun Gao, Renrui Zhang, Jian Zhang
分類: cs.CV, cs.RO
原文アブストラクト
The scale and quality of point cloud datasets constrain the advancement of point cloud learning. Recently, with the development of multi-modal learning, the incorporation of domain-agnostic prior knowledge from other modalities, such as images and text, to assist in point cloud feature learning has been considered a promising avenue. Existing methods have demonstrated the effectiveness of multi-modal contrastive training and feature distillation on point clouds. However, challenges remain, including the requirement for paired triplet data, redundancy and ambiguity in supervised features, and the disruption of the original priors. In this paper, we propose a language-assisted approach to point cloud feature learning (LAST-PCL), enriching semantic concepts through LLMs-based text enrichment. We achieve de-redundancy and feature dimensionality reduction without compromising textual priors by statistical-based and training-free significant feature selection. Furthermore, we also delve into an in-depth analysis of the impact of text contrastive training on the point cloud. Extensive experiments validate that the proposed method learns semantically meaningful point cloud features and achieves state-of-the-art or comparable performance in 3D semantic segmentation, 3D object detection, and 3D scene classification tasks.
