ロボットとのマルチモーダルな社会的会話に向けて:視覚言語モデルの活用
Towards Multimodal Social Conversations with Robots: Using Vision-Language Models
社会的ロボットが視覚情報を活用してマルチモーダルな会話を行うための要件を整理し、視覚言語モデルを適応させる方法と課題を論じた論文。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Ruben Janssens, Tony Belpaeme
分類: cs.RO, cs.CL, cs.HC
原文アブストラクト
Large language models have given social robots the ability to autonomously engage in open-domain conversations. However, they are still missing a fundamental social skill: making use of the multiple modalities that carry social interactions. While previous work has focused on task-oriented interactions that require referencing the environment or specific phenomena in social interactions such as dialogue breakdowns, we outline the overall needs of a multimodal system for social conversations with robots. We then argue that vision-language models are able to process this wide range of visual information in a sufficiently general manner for autonomous social robots. We describe how to adapt them to this setting, which technical challenges remain, and briefly discuss evaluation practices.
