BlabberSeg: リアルタイム組み込み向けオープンボキャブラリ航空画像セグメンテーション
BlabberSeg: Real-Time Embedded Open-Vocabulary Aerial Segmentation
CLIPSegを最適化し、プロンプトとモデル特徴を再利用することで、UAV搭載のJetson上でオープンボキャブラリ航空セグメンテーションを約16.8Hzでリアルタイム実行可能にした。
著者: Haechan Mark Bong, Ricardo de Azambuja, Giovanni Beltrame
分類: cs.RO
原文アブストラクト
Real-time aerial image segmentation plays an important role in the environmental perception of Uncrewed Aerial Vehicles (UAVs). We introduce BlabberSeg, an optimized Vision-Language Model built on CLIPSeg for on-board, real-time processing of aerial images by UAVs. BlabberSeg improves the efficiency of CLIPSeg by reusing prompt and model features, reducing computational overhead while achieving real-time open-vocabulary aerial segmentation. We validated BlabberSeg in a safe landing scenario using the Dynamic Open-Vocabulary Enhanced SafE-Landing with Intelligence (DOVESEI) framework, which uses visual servoing and open-vocabulary segmentation. BlabberSeg reduces computational costs significantly, with a speed increase of 927.41% (16.78 Hz) on a NVIDIA Jetson Orin AGX (64GB) compared with the original CLIPSeg (1.81Hz), achieving real-time aerial segmentation with negligible loss in accuracy (2.1% as the ratio of the correctly segmented area with respect to CLIPSeg). BlabberSeg's source code is open and available online.
