Rank2Tell: 重要度ランキングと理由付けのためのマルチモーダル運転データセット
Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoning
複雑な交通シーンで重要物体の重要度ランキングとその理由を説明するための、マルチモーダルな自己中心視点データセットを構築し、ランキングと自然言語キャプション生成を同時に行うモデルでベンチマークした。
著者: Enna Sachdeva, Nakul Agarwal, Suhas Chundi, Sean Roelofs, Jiachen Li, Mykel Kochenderfer, Chiho Choi, Behzad Dariush
分類: cs.CV, cs.AI, cs.LG, cs.RO
原文アブストラクト
The widespread adoption of commercial autonomous vehicles (AVs) and advanced driver assistance systems (ADAS) may largely depend on their acceptance by society, for which their perceived trustworthiness and interpretability to riders are crucial. In general, this task is challenging because modern autonomous systems software relies heavily on black-box artificial intelligence models. Towards this goal, this paper introduces a novel dataset, Rank2Tell, a multi-modal ego-centric dataset for Ranking the importance level and Telling the reason for the importance. Using various close and open-ended visual question answering, the dataset provides dense annotations of various semantic, spatial, temporal, and relational attributes of various important objects in complex traffic scenarios. The dense annotations and unique attributes of the dataset make it a valuable resource for researchers working on visual scene understanding and related fields. Furthermore, we introduce a joint model for joint importance level ranking and natural language captions generation to benchmark our dataset and demonstrate performance with quantitative evaluations.
