ニューラルニューラルテクスチャによるSim2Realの一貫性実現
Neural Neural Textures Make Sim2Real Consistent
微分可能レンダリングと画像変換を組み合わせ、ニューラルテクスチャを用いて時間的一貫性を保ちながらSim2Realを実現するTRITONを提案。
著者: Ryan Burgert, Jinghuan Shang, Xiang Li, Michael Ryoo
分類: cs.CV, cs.GR, cs.LG, cs.RO
原文アブストラクト
Unpaired image translation algorithms can be used for sim2real tasks, but many fail to generate temporally consistent results. We present a new approach that combines differentiable rendering with image translation to achieve temporal consistency over indefinite timescales, using surface consistency losses and \emph{neural neural textures}. We call this algorithm TRITON (Texture Recovering Image Translation Network): an unsupervised, end-to-end, stateless sim2real algorithm that leverages the underlying 3D geometry of input scenes by generating realistic-looking learnable neural textures. By settling on a particular texture for the objects in a scene, we ensure consistency between frames statelessly. Unlike previous algorithms, TRITON is not limited to camera movements -- it can handle the movement of objects as well, making it useful for downstream tasks such as robotic manipulation.
