自己回帰拡散モデルによる制御可能な動作生成と再構成
Controllable Motion Synthesis and Reconstruction with Autoregressive Diffusion Models
他のモダリティの制御コンテキストに条件付けられた自己回帰拡散モデルMoDiffを提案し、多様な動作生成と不完全な姿勢の再構成を実現した。
著者: Wenjie Yin, Ruibo Tu, Hang Yin, Danica Kragic, Hedvig Kjellström, Mårten Björkman
分類: cs.CV, cs.LG
原文アブストラクト
Data-driven and controllable human motion synthesis and prediction are active research areas with various applications in interactive media and social robotics. Challenges remain in these fields for generating diverse motions given past observations and dealing with imperfect poses. This paper introduces MoDiff, an autoregressive probabilistic diffusion model over motion sequences conditioned on control contexts of other modalities. Our model integrates a cross-modal Transformer encoder and a Transformer-based decoder, which are found effective in capturing temporal correlations in motion and control modalities. We also introduce a new data dropout method based on the diffusion forward process to provide richer data representations and robust generation. We demonstrate the superior performance of MoDiff in controllable motion synthesis for locomotion with respect to two baselines and show the benefits of diffusion data dropout for robust synthesis and reconstruction of high-fidelity motion close to recorded data.
