ローリングシャッター映像のためのイベントガイド付きビデオフレーム補間の自己教師あり学習
Self-supervised Learning of Event-guided Video Frame Interpolation for Rolling Shutter Frames
ローリングシャッターカメラとイベントカメラを組み合わせ、歪みのない高フレームレートのグローバルシャッター映像を自己教師あり学習で復元する手法を提案した。
著者: Yunfan Lu, Guoqiang Liang, Yiran Shen, Lin Wang
分類: cs.CV, cs.RO
原文アブストラクト
Most consumer cameras use rolling shutter (RS) exposure, which often leads to distortions such as skew and jelly effects. These videos are further limited by bandwidth and frame rate constraints. In this paper, we explore the potential of event cameras, which offer high temporal resolution. We propose a framework to recover global shutter (GS) high-frame-rate videos without RS distortion by combining an RS camera and an event camera. Due to the lack of real-world datasets, our framework adopts a self-supervised strategy based on a displacement field, a dense 3D spatiotemporal representation of pixel motion during exposure. This enables mutual reconstruction between RS and GS frames and facilitates slow-motion recovery. We combine RS frames with the displacement field to generate GS frames, and integrate inverse mapping and RS frame warping for self-supervision. Experiments on four datasets show that our method removes distortion, reduces bandwidth usage by 94 percent, and achieves 16 ms per frame at 32x interpolation.
