微分可能シミュレーションによる物性推定と視覚運動制御
gradSim: Differentiable simulation for system identification and visuomotor control
微分可能なマルチフィジックスシミュレーションと微分可能レンダリングを組み合わせ、動画のピクセルから直接、質量・摩擦・弾性などの物理特性を推定するフレームワークgradSimを提案。3D教師信号なしで視覚運動制御タスクの学習も可能にする。
著者: Krishna Murthy Jatavallabhula, Miles Macklin, Florian Golemo, Vikram Voleti, Linda Petrini, Martin Weiss, Breandan Considine, Jerome Parent-Levesque, Kevin Xie, Kenny Erleben, Liam Paull, Florian Shkurti, Derek Nowrouzezahrai, Sanja Fidler
分類: cs.CV, cs.AI, cs.LG, cs.RO
原文アブストラクト
We consider the problem of estimating an object's physical properties such as mass, friction, and elasticity directly from video sequences. Such a system identification problem is fundamentally ill-posed due to the loss of information during image formation. Current solutions require precise 3D labels which are labor-intensive to gather, and infeasible to create for many systems such as deformable solids or cloth. We present gradSim, a framework that overcomes the dependence on 3D supervision by leveraging differentiable multiphysics simulation and differentiable rendering to jointly model the evolution of scene dynamics and image formation. This novel combination enables backpropagation from pixels in a video sequence through to the underlying physical attributes that generated them. Moreover, our unified computation graph -- spanning from the dynamics and through the rendering process -- enables learning in challenging visuomotor control tasks, without relying on state-based (3D) supervision, while obtaining performance competitive to or better than techniques that rely on precise 3D labels.
