GCC-PHAT特徴を用いたBatVision：音から視覚への予測精度向上
BatVision with GCC-PHAT Features for Better Sound to Vision Predictions
バイノーラルエコーから深度マップとグレースケール画像を生成するGANベースのBatVisionを改良し、生波形の代わりにGCC-PHAT特徴を入力とすることで予測精度と知覚品質を向上させた。
著者: Jesper Haahr Christensen, Sascha Hornauer, Stella Yu
分類: cs.CV, cs.RO, cs.SD, eess.AS
原文アブストラクト
Inspired by sophisticated echolocation abilities found in nature, we train a generative adversarial network to predict plausible depth maps and grayscale layouts from sound. To achieve this, our sound-to-vision model processes binaural echo-returns from chirping sounds. We build upon previous work with BatVision that consists of a sound-to-vision model and a self-collected dataset using our mobile robot and low-cost hardware. We improve on the previous model by introducing several changes to the model, which leads to a better depth and grayscale estimation, and increased perceptual quality. Rather than using raw binaural waveforms as input, we generate generalized cross-correlation (GCC) features and use these as input instead. In addition, we change the model generator and base it on residual learning and use spectral normalization in the discriminator. We compare and present both quantitative and qualitative improvements over our previous BatVision model.
