日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

データセット/手と物体のインタラクションarXiv:2603.28760

SHOW3D: 実世界での3D手と物体のシーンキャプチャ

SHOW3D: Capturing Scenes of 3D Hands and Objects in the Wild

シェア:XThreadsFacebookLINEはてブBluesky

制約のない実環境で手と物体の正確な3Dアノテーションを生成できる新しいマーカーレス多カメラシステムを提案し、多様な実世界環境での手と物体のインタラクションを含む大規模データセットSHOW3Dを構築した。

詳しい要約

1. どんなもの?

- 3D hand-object interaction dataset in the wild - Marker-less multi-camera system for capturing scenes - Lightweight back-mounted rig synchronized with VR headset - Ego-exo tracking pipeline for 3D annotations - Large-scale dataset with diverse real-world environments

2. 先行研究と比べてどこがすごい?

- Existing datasets captured in controlled studio settings - Limited environmental diversity and generalization - SHOW3D enables nearly unconstrained mobility in the wild - Reduces trade-off between realism and annotation accuracy - First large-scale dataset with 3D annotations in outdoor settings

3. 技術・手法の肝は?

- Multi-camera system: back-mounted, lightweight, synchronized with VR headset - Marker-less capture for unconstrained mobility - Ego-exo tracking pipeline for 3D ground-truth annotation - Rigorous evaluation of annotation quality - Calibration and synchronization between cameras and VR headset

4. どうやって有効だと検証した?

- Experiments on several downstream tasks - Rigorous evaluation of tracking pipeline quality - Validation of reduced trade-off between realism and accuracy - Dataset quality assessed through downstream task performance

5. 議論はある?

- Addresses limitations of controlled studio settings - Enables generalization to real-world scenarios - Reduces fundamental trade-off between environmental realism and annotation accuracy - Potential for diverse applications in egocentric computer vision

6. 次に読むべき論文は?

- Ego-exo tracking methods - Hand-object interaction datasets (e.g., in controlled settings) - Multi-camera calibration and synchronization techniques - VR headset integration for data capture

※ AIが要旨から生成した要約です。正確性は原文をご確認ください。

著者: Patrick Rim, Kevin Harris, Braden Copple, Shangchen Han, Xu Xie, Ivan Shugurov, Sizhe An, He Wen, Alex Wong, Tomas Hodan, Kun He

分類: cs.CV, cs.RO

原文アブストラクト

Accurate 3D understanding of human hands and objects during manipulation remains a significant challenge for egocentric computer vision. Existing hand-object interaction datasets are predominantly captured in controlled studio settings, which limits both environmental diversity and the ability of models trained on such data to generalize to real-world scenarios. To address this challenge, we introduce a novel marker-less multi-camera system that allows for nearly unconstrained mobility in genuinely in-the-wild conditions, while still having the ability to generate precise 3D annotations of hands and objects. The capture system consists of a lightweight, back-mounted, multi-camera rig that is synchronized and calibrated with a user-worn VR headset. For 3D ground-truth annotation of hands and objects, we develop an ego-exo tracking pipeline and rigorously evaluate its quality. Finally, we present SHOW3D, the first large-scale dataset with 3D annotations that show hands interacting with objects in diverse real-world environments, including outdoor settings. Our approach significantly reduces the fundamental trade-off between environmental realism and accuracy of 3D annotations, which we validate with experiments on several downstream tasks. show3d-dataset.github.io

  1. 1Boston Dynamics、車輪付きAtlasの研究試作機を紹介研究・技術
  2. 2GXOとApptronik、Apolloの物流現場試験を拡大 最大1,000台の展開を構想量産・展開
  3. 3newmoとKDDIが資本業務提携 2028年に堺市で自動運転タクシー商用化目指す提携・買収
  4. 4NVIDIA、CoRLでGR00T 2とIsaac Lab 3.0を発表新製品・発表
  5. 5UC BerkeleyとNVIDIAの研究者、AIエージェントでロボット制御コードを仮想環境で改善研究・技術

PR・アフィリエイト広告

OZ GAMING(GPU搭載BTOゲーミングPC)

OZ GAMING(GPU搭載BTOゲーミングPC)の公式広告バナーOZ GAMINGで完成品PCを見る ↗
PR本紙発行元 EmplifAI