単一の実演動画から撮影スタイルを模倣するワンショット撮影学習
One-Shot Imitation Filming of Human Motion Videos
同じスタイルの実演動画を1本見るだけで撮影スタイルを模倣し、6自由度ドローンで新たな状況に転移して撮影できるフレームワークを提案した。
著者: Chong Huang, Yuanjie Dang, Peng Chen, Xin Yang, Kwang-Ting, Cheng
分類: cs.CV, cs.RO
原文アブストラクト
Imitation learning has been applied to mimic the operation of a human cameraman in several autonomous cinematography systems. To imitate different filming styles, existing methods train multiple models, where each model handles a particular style and requires a significant number of training samples. As a result, existing methods can hardly generalize to unseen styles. In this paper, we propose a framework, which can imitate a filming style by "seeing" only a single demonstration video of the same style, i.e., one-shot imitation filming. This is done by two key enabling techniques: 1) feature extraction of the filming style from the demo video, and 2) filming style transfer from the demo video to the new situation. We implement the approach with deep neural network and deploy it to a 6 degrees of freedom (DOF) real drone cinematography system by first predicting the future camera motions, and then converting them to the drone's control commands via an odometer. Our experimental results on extensive datasets and showcases exhibit significant improvements in our approach over conventional baselines and our approach can successfully mimic the footage with an unseen style.
