海上マルチシーン認識のための軽量マルチモーダルAIフレームワーク
Lightweight Multimodal Artificial Intelligence Framework for Maritime Multi-Scene Recognition
画像・テキスト・分類ベクトルを統合したマルチモーダルAIで海上シーンを高精度認識し、量子化により軽量化して自律表面車両への実装を可能にした。
著者: Xinyu Xi, Hua Yang, Shentai Zhang, Yijie Liu, Sijin Sun, Xiuju Fu
分類: cs.CV, cs.AI, cs.LG, cs.RO
原文アブストラクト
Maritime Multi-Scene Recognition is crucial for enhancing the capabilities of intelligent marine robotics, particularly in applications such as marine conservation, environmental monitoring, and disaster response. However, this task presents significant challenges due to environmental interference, where marine conditions degrade image quality, and the complexity of maritime scenes, which requires deeper reasoning for accurate recognition. Pure vision models alone are insufficient to address these issues. To overcome these limitations, we propose a novel multimodal Artificial Intelligence (AI) framework that integrates image data, textual descriptions and classification vectors generated by a Multimodal Large Language Model (MLLM), to provide richer semantic understanding and improve recognition accuracy. Our framework employs an efficient multimodal fusion mechanism to further enhance model robustness and adaptability in complex maritime environments. Experimental results show that our model achieves 98$\%$ accuracy, surpassing previous SOTA models by 3.5$\%$. To optimize deployment on resource-constrained platforms, we adopt activation-aware weight quantization (AWQ) as a lightweight technique, reducing the model size to 68.75MB with only a 0.5$\%$ accuracy drop while significantly lowering computational overhead. This work provides a high-performance solution for real-time maritime scene recognition, enabling Autonomous Surface Vehicles (ASVs) to support environmental monitoring and disaster response in resource-limited settings.
関連論文
- AeroIntent: UAVミッション意図認識のための同期マルチモーダルベンチマークマルチモーダル認識
- MultiFly: 実世界マルチモーダル航空データセットと注釈効率の良いラベル転送およびクロスモーダル意味整合性マルチモーダル認識
- GlassFormer: レーダーと深度の融合によるリアルタイムガラスセグメンテーションの学習マルチモーダル認識
- 対数尤度比融合によるドローン・移動ロボット遠隔操作のための解釈可能なマルチモーダルジェスチャ認識マルチモーダル認識
- 照明条件に応じてRGBと赤外線を適応融合する自律追尾タレットシステムの評価マルチモーダル認識
- OmniUnet: RGB・深度・熱画像を用いた惑星探査ローバー向け非構造地形セグメンテーションのマルチモーダルネットワークマルチモーダル認識
