歩容から感情を読み取る時空間グラフ畳み込みネットワークSTEP
STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits
歩行映像の歩容特徴を時空間グラフ畳み込みネットワークで解析し、喜び・悲しみ・怒り・中立の4感情を分類する手法を提案。生成ネットワークで合成歩容を拡張し、E-Gaitデータセットも公開した。
著者: Uttaran Bhattacharya, Trisha Mittal, Rohan Chandra, Tanmay Randhavane, Aniket Bera, Dinesh Manocha
分類: cs.CV, cs.RO
原文アブストラクト
We present a novel classifier network called STEP, to classify perceived human emotion from gaits, based on a Spatial Temporal Graph Convolutional Network (ST-GCN) architecture. Given an RGB video of an individual walking, our formulation implicitly exploits the gait features to classify the emotional state of the human into one of four emotions: happy, sad, angry, or neutral. We use hundreds of annotated real-world gait videos and augment them with thousands of annotated synthetic gaits generated using a novel generative network called STEP-Gen, built on an ST-GCN based Conditional Variational Autoencoder (CVAE). We incorporate a novel push-pull regularization loss in the CVAE formulation of STEP-Gen to generate realistic gaits and improve the classification accuracy of STEP. We also release a novel dataset (E-Gait), which consists of $2,177$ human gaits annotated with perceived emotions along with thousands of synthetic gaits. In practice, STEP can learn the affective features and exhibits classification accuracy of 89% on E-Gait, which is 14 - 30% more accurate over prior methods.
