異種マルチモーダルLLMエージェントによる物体SLAMの意味強化
Semantic Enhancement for Object SLAM with Heterogeneous Multimodal Large Language Model Agents
複数のマルチモーダルLLMエージェントを統合し、非同期処理と意味・マハラノビス距離を組み合わせたデータ関連付けで物体SLAMの意味マッピング精度を向上させるフレームワークを提案。
著者: Jungseok Hong, Ran Choi, John J. Leonard
分類: cs.RO
原文アブストラクト
Object Simultaneous Localization and Mapping (SLAM) systems struggle to correctly associate semantically similar objects in close proximity, especially in cluttered indoor environments and when scenes change. We present Semantic Enhancement for Object SLAM (SEO-SLAM), a novel framework that enhances semantic mapping by integrating heterogeneous multimodal large language model (MLLM) agents. Our method enables scene adaptation while maintaining a semantically rich map. To improve computational efficiency, we propose an asynchronous processing scheme that significantly reduces the agents' inference time without compromising semantic accuracy or SLAM performance. Additionally, we introduce a multi-data association strategy using a cost matrix that combines semantic and Mahalanobis distances, formulating the problem as a Linear Assignment Problem (LAP) to alleviate perceptual aliasing. Experimental results demonstrate that SEO-SLAM consistently achieves higher semantic accuracy and reduces false positives compared to baselines, while our asynchronous MLLM agents significantly improve processing efficiency over synchronous setups. We also demonstrate that SEO-SLAM has the potential to improve downstream tasks such as robotic assistance. Our dataset is publicly available at: jungseokhong.com/SEO-SLAM.
