認知型TransFuser:意味論誘導トランスフォーマーによるセンサフュージョンとウェイポイント予測の改善
Cognitive TransFuser: Semantics-guided Transformer-based Sensor Fusion for Improved Waypoint Prediction
交通信号認識や意味セグメンテーションなどの補助タスクを活用し、RGBとLiDARを融合するマルチタスクTransformerネットワークで、自動運転のウェイポイント予測精度を向上させた研究。
著者: Hwan-Soo Choi, Jongoh Jeong, Young Hoo Cho, Kuk-Jin Yoon, Jong-Hwan Kim
分類: cs.RO, cs.AI
原文アブストラクト
Sensor fusion approaches for intelligent self-driving agents remain key to driving scene understanding given visual global contexts acquired from input sensors. Specifically, for the local waypoint prediction task, single-modality networks are still limited by strong dependency on the sensitivity of the input sensor, and thus recent works therefore promote the use of multiple sensors in fusion in feature level in practice. While it is well known that multiple data modalities encourage mutual contextual exchange, it requires global 3D scene understanding in real-time with minimal computation upon deployment to practical driving scenarios, thereby placing greater significance on the training strategy given a limited number of practically usable sensors. In this light, we exploit carefully selected auxiliary tasks that are highly correlated with the target task of interest (e.g., traffic light recognition and semantic segmentation) by fusing auxiliary task features and also using auxiliary heads for waypoint prediction based on imitation learning. Our RGB-LIDAR-based multi-task feature fusion network, coined Cognitive TransFuser, augments and exceeds the baseline network by a significant margin for safer and more complete road navigation in the CARLA simulator. We validate the proposed network on the Town05 Short and Town05 Long Benchmark through extensive experiments, achieving up to 44.2 FPS real-time inference time.
関連論文
- 見えない轍を追う:熱赤外・RGB融合による自己教師ありホイールトラック検出自動運転/センサフュージョン
- レーダー画像融合による動的物体セグメンテーションと自己運動推定自動運転/センサフュージョン
- TransFuser:トランスフォーマーによるセンサフュージョンを用いた自動運転のための模倣学習自動運転/センサフュージョン
- ADASにおけるマルチセンサフュージョン誤りの検出自動運転/センサフュージョン
