視覚運動協調の予測学習
Learning Predictive Visuomotor Coordination
一人称視点の映像と身体運動から頭部姿勢・視線・上半身の動きを予測する拡散ベースのモデルを提案し、大規模データセットで汎化性能を示した。
著者: Wenqi Jia, Bolin Lai, Miao Liu, Danfei Xu, James M. Rehg
分類: cs.CV, cs.RO
原文アブストラクト
Understanding and predicting human visuomotor coordination is crucial for applications in robotics, human-computer interaction, and assistive technologies. This work introduces a forecasting-based task for visuomotor modeling, where the goal is to predict head pose, gaze, and upper-body motion from egocentric visual and kinematic observations. We propose a \textit{Visuomotor Coordination Representation} (VCR) that learns structured temporal dependencies across these multimodal signals. We extend a diffusion-based motion modeling framework that integrates egocentric vision and kinematic sequences, enabling temporally coherent and accurate visuomotor predictions. Our approach is evaluated on the large-scale EgoExo4D dataset, demonstrating strong generalization across diverse real-world activities. Our results highlight the importance of multimodal integration in understanding visuomotor coordination, contributing to research in visuomotor learning and human behavior modeling. Project Page: https://vjwq.github.io/VCR/.
