日本フィジカルAI新聞

世界のフィジカルAIを、日本語で。

感情認識arXiv:2409.10687

Vision Transformerを用いた人間-ロボット対話における個人適応型音声感情認識

Personalized Speech Emotion Recognition in Human-Robot Interaction using Vision Transformers

シェア:XThreadsFacebookLINEはてブBluesky

NAOロボットとの会話音声データを用い、ViT/BEiTをベンチマークで微調整・アンサンブルすることで、個人の話し方に適応した4感情(中性・喜び・悲しみ・怒り)の音声感情認識を高精度化した研究。

著者: Ruchik Mishra, Andrew Frye, Madan Mohan Rayguru, Dan O. Popa

分類: eess.AS, cs.HC, cs.RO, cs.SD

原文アブストラクト

Emotions are an essential element in verbal communication, so understanding individuals' affect during a human-robot interaction (HRI) becomes imperative. This paper investigates the application of vision transformer models, namely ViT (Vision Transformers) and BEiT (BERT Pre-Training of Image Transformers) pipelines, for Speech Emotion Recognition (SER) in HRI. The focus is to generalize the SER models for individual speech characteristics by fine-tuning these models on benchmark datasets and exploiting ensemble methods. For this purpose, we collected audio data from different human subjects having pseudo-naturalistic conversations with the NAO robot. We then fine-tuned our ViT and BEiT-based models and tested these models on unseen speech samples from the participants. In the results, we show that fine-tuning vision transformers on benchmark datasets and and then using either these already fine-tuned models or ensembling ViT/BEiT models gets us the highest classification accuracies per individual when it comes to identifying four primary emotions from their speech: neutral, happy, sad, and angry, as compared to fine-tuning vanilla-ViTs or BEiTs.

関連論文

  1. 1パナソニックHDなど、触覚に即応する両手作業AI「T-Rex」を共同開発研究・技術
  2. 2川崎重工業のロボット外販を報道新製品・発表
  3. 3Boston Dynamics、GlobalFoundriesでAtlasを試験へ 半導体製造で2027年初頭から半導体・GPU
  4. 4丸文、RealManのロボットアームやヒューマノイドを国内販売提携・買収
  5. 5NVIDIA、RTX Spark小型デスクトップを発売 ASUS、Dell、MSIから半導体・GPU

PR・アフィリエイト広告

FRONTIER(GPU搭載BTOパソコン)

FRONTIER(GPU搭載BTOパソコン)の公式広告バナーFRONTIERでGPU搭載PCを見る ↗
PR本紙発行元 EmplifAI