日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

ビジュアルサーボ/ドローン追跡arXiv:2405.11655

任意対象追跡ドローンTAR:基盤モデルとビジュアルサーボによるマルチモーダル追跡

Track Anything Rapter(TAR)

シェア:XThreadsFacebookLINEはてブBluesky

DINO・CLIP・SAMなどの基盤モデルを組み合わせ、テキスト・画像・クリックといったマルチモーダルな指示に基づいて対象を検出・セグメンテーションし、ビジュアルサーボ制御でドローンが安定追従するシステムを構築・検証した。

著者: Tharun V. Puthanveettil, Fnu Obaid ur Rahman

分類: cs.CV, cs.AI, cs.RO

原文アブストラクト

Object tracking is a fundamental task in computer vision with broad practical applications across various domains, including traffic monitoring, robotics, and autonomous vehicle tracking. In this project, we aim to develop a sophisticated aerial vehicle system known as Track Anything Rapter (TAR), designed to detect, segment, and track objects of interest based on user-provided multimodal queries, such as text, images, and clicks. TAR utilizes cutting-edge pre-trained models like DINO, CLIP, and SAM to estimate the relative pose of the queried object. The tracking problem is approached as a Visual Servoing task, enabling the UAV to consistently focus on the object through advanced motion planning and control algorithms. We showcase how the integration of these foundational models with a custom high-level control algorithm results in a highly stable and precise tracking system deployed on a custom-built PX4 Autopilot-enabled Voxl2 M500 drone. To validate the tracking algorithm's performance, we compare it against Vicon-based ground truth. Additionally, we evaluate the reliability of the foundational models in aiding tracking in scenarios involving occlusions. Finally, we test and validate the model's ability to work seamlessly with multiple modalities, such as click, bounding box, and image templates.

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2newmoとKDDIが資本業務提携 2028年に堺市で自動運転タクシー商用化目指す提携・買収
  3. 3Google DeepMind、ロボット向けAIを発表新製品・発表
  4. 4NVIDIA、CoRLでGR00T 2とIsaac Lab 3.0を発表新製品・発表
  5. 5GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開

PR・アフィリエイト広告

FRONTIER(GPU搭載BTOパソコン)

FRONTIER(GPU搭載BTOパソコン)の公式広告バナーFRONTIERでGPU搭載PCを見る ↗
PR本紙発行元 EmplifAI