マルチモーダル大規模言語モデルによる実時間状況推論
Multimodal Large Language Models for Real-Time Situated Reasoning
GPT-4oとTurtleBot 4を組み合わせ、家庭内の視覚情報から掃除開始の適切さを判断するシステムを構築し、文脈や価値観に基づく意思決定の可能性と課題を示した。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Giulio Antonio Abbo, Senne Lenaerts, Tony Belpaeme
分類: cs.RO
原文アブストラクト
In this work, we explore how multimodal large language models can support real-time context- and value-aware decision-making. To do so, we combine the GPT-4o language model with a TurtleBot 4 platform simulating a smart vacuum cleaning robot in a home. The model evaluates the environment through vision input and determines whether it is appropriate to initiate cleaning. The system highlights the ability of these models to reason about domestic activities, social norms, and user preferences and take nuanced decisions aligned with the values of the people involved, such as cleanliness, comfort, and safety. We demonstrate the system in a realistic home environment, showing its ability to infer context and values from limited visual input. Our results highlight the promise of multimodal large language models in enhancing robotic autonomy and situational awareness, while also underscoring challenges related to consistency, bias, and real-time performance.
