自動運転における歩行者行動とシーン理解への視覚言語モデルの応用
Application of Vision-Language Model to Pedestrians Behavior and Scene Understanding in Autonomous Driving
大規模視覚言語モデルの知識を効率的な視覚ネットワークに蒸留し、歩行者の行動予測とシーン理解に適用することで、多様で包括的な意味属性の生成と予測精度の向上を実現した。
著者: Haoxiang Gao, Li Zhang, Yu Zhao, Zhou Yang, Jinghan Cao
分類: cs.CV, cs.AI, cs.LG, cs.RO
原文アブストラクト
Vision-language models (VLMs) have become a promising approach to enhancing perception and decision-making in autonomous driving. The gap remains in applying VLMs to understand complex scenarios interacting with pedestrians and efficient vehicle deployment. In this paper, we propose a knowledge distillation method that transfers knowledge from large-scale vision-language foundation models to efficient vision networks, and we apply it to pedestrian behavior prediction and scene understanding tasks, achieving promising results in generating more diverse and comprehensive semantic attributes. We also utilize multiple pre-trained models and ensemble techniques to boost the model's performance. We further examined the effectiveness of the model after knowledge distillation; the results show significant metric improvements in open-vocabulary perception and trajectory prediction tasks, which can potentially enhance the end-to-end performance of autonomous driving.
