日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

3D再構成arXiv:2611.21302

Déjà-Events: イベントカメラによるスパース性対応適応ゲーティングを用いたマルチビュー3D幾何推定

Déjà-Events: Multi-view 3D Geometry with Event Cameras via Sparsity-Aware Adaptive Gating

シェア:XThreadsFacebookLINEはてブBluesky

RGB教師モデルから蒸留したイベントカメラ専用モデルで、複数視点から深度・カメラ姿勢・ポイントマップを単一フォワードパスで推定する。イベント密度に応じた適応ゲーティングでスパース領域の精度低下を抑え、実データでも高精度を達成。

著者: Arda Cinar Demirtas, Pablo Varela Bernal, Marc Pollefeys, Sunghwan Hong

分類: cs.CV, cs.RO

原文アブストラクト

Event cameras keep capturing structure where conventional cameras blur or saturate, which makes them attractive for 3D perception in fast or poorly lit scenes. Event datasets with dense 3D ground truth, however, are far smaller and less diverse than the image collections behind feed-forward 3D reconstruction models. We present Déjà-Events, an event-only model that predicts depth, camera pose and pointmaps from multiple views in a single forward pass. Rather than relying on scarce 3D labels, we distil it from Déjà View, a feed-forward RGB model that refines its tokens by repeatedly applying one shared transformer block. On paired RGB and event recordings, the event student learns to reproduce the frozen teacher's final tokens, which requires no depth or pose ground truth and trains only about 1% of its parameters. Because events are dense at edges and moving texture but nearly absent in textureless regions, we further introduce sparsity-aware adaptive gating, which adapts the shared block's gates to events and adjusts them per patch by event density, so that accuracy holds as refinement steps are added. Trained only on simulated data, Déjà-Events achieves the lowest pointmap error among the event-based methods we evaluate, on EventScape and zero-shot on the real MVSEC and DSEC datasets. On DSEC it halves the error of the strongest baseline (0.15 vs. 0.30 AbsRel) with about a tenth of its parameters. It also has the lowest depth error on EventScape and DSEC and the lowest trajectory error on EventScape.

関連論文

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2Google DeepMind、ロボット向けAIを発表新製品・発表
  3. 3GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開
  4. 4パナソニックHDなど、触覚に即応する両手作業AI「T-Rex」を共同開発研究・技術
  5. 5SoftBank主導、Agile Robotsが5億ドル調達 NVIDIAと日立も参加資金調達

PR・アフィリエイト広告

OZ GAMING(GPU搭載BTOゲーミングPC)

OZ GAMING(GPU搭載BTOゲーミングPC)の公式広告バナーOZ GAMINGで完成品PCを見る ↗
PR本紙発行元 EmplifAI