SplaTraj: セマンティックGaussian Splattingによるカメラ軌道生成
SplaTraj: Camera Trajectory Generation with Semantic Gaussian Splatting
言語指示に基づき、Gaussian Splattingで再構成した環境内を映える形で巡るカメラ軌道を連続時間最適化で生成するフレームワーク。
著者: Xinyi Liu, Tianyi Zhang, Matthew Johnson-Roberson, Weiming Zhi
分類: cs.RO, cs.AI, cs.CV, cs.LG
原文アブストラクト
Many recent developments for robots to represent environments have focused on photorealistic reconstructions. This paper particularly focuses on generating sequences of images from the photorealistic Gaussian Splatting models, that match instructions that are given by user-inputted language. We contribute a novel framework, SplaTraj, which formulates the generation of images within photorealistic environment representations as a continuous-time trajectory optimization problem. Costs are designed so that a camera following the trajectory poses will smoothly traverse through the environment and render the specified spatial information in a photogenic manner. This is achieved by querying a photorealistic representation with language embedding to isolate regions that correspond to the user-specified inputs. These regions are then projected to the camera's view as it moves over time and a cost is constructed. We can then apply gradient-based optimization and differentiate through the rendering to optimize the trajectory for the defined cost. The resulting trajectory moves to photogenically view each of the specified objects. We empirically evaluate our approach on a suite of environments and instructions, and demonstrate the quality of generated image sequences.
