MambaPlace: 注意機構付きMambaによるテキスト・点群クロスモーダル位置認識
MambaPlace:Text-to-Point-Cloud Cross-Modal Place Recognition with Attention Mamba Mechanisms
テキスト記述と3D点群をMambaベースの注意機構で融合し、ロボットの位置認識を粗から細へと高精度化するフレームワークを提案。
著者: Tianyi Shang, Zhenyu Li, Pengjie Xu, Jinwei Qiao
分類: cs.CV, cs.RO
原文アブストラクト
Vision Language Place Recognition (VLVPR) enhances robot localization performance by incorporating natural language descriptions from images. By utilizing language information, VLVPR directs robot place matching, overcoming the constraint of solely depending on vision. The essence of multimodal fusion lies in mining the complementary information between different modalities. However, general fusion methods rely on traditional neural architectures and are not well equipped to capture the dynamics of cross modal interactions, especially in the presence of complex intra modal and inter modal correlations. To this end, this paper proposes a novel coarse to fine and end to end connected cross modal place recognition framework, called MambaPlace. In the coarse localization stage, the text description and 3D point cloud are encoded by the pretrained T5 and instance encoder, respectively. They are then processed using Text Attention Mamba (TAM) and Point Clouds Mamba (PCM) for data enhancement and alignment. In the subsequent fine localization stage, the features of the text description and 3D point cloud are cross modally fused and further enhanced through cascaded Cross Attention Mamba (CCAM). Finally, we predict the positional offset from the fused text point cloud features, achieving the most accurate localization. Extensive experiments show that MambaPlace achieves improved localization accuracy on the KITTI360Pose dataset compared to the state of the art methods.
関連論文
- 地形正規化垂直分布による森林LiDAR位置認識のスキャンコンテキスト失敗診断位置認識
- UpDown-SC: 重力正規化された上下包絡線スキャンコンテキストによる屋内LiDAR位置認識位置認識
- M3GA-Wild: 森林におけるマルチモーダル・マルチセッションの地上-空中位置認識のための大規模データセットとベンチマーク位置認識
- DXPR: 視覚基盤モデルを用いた深度ベースの視覚-LiDARクロスモーダル位置認識位置認識
- InLiER: 中間混合基数構造キーポイントトークン化による学習不要の異種LiDAR位置認識位置認識
- G-PROBE: 3D点群のためのクロスFOV位置認識と確信度結合型ローカライゼーション位置認識
