PAg-NeRF: 農業ロボティクス向けの高速で効率的なエンドツーエンドパノプティック3D表現
PAg-NeRF: Towards fast and efficient end-to-end panoptic 3D representations for agricultural robotics
ノイズの多いロボットのオドメトリとフレーム間でIDが不整合なパノプティック予測を用いて学習し、一貫したインスタンスIDを持つ3Dパノプティック表現を出力するNeRFベースのシステムを提案。園芸シナリオで評価し、ベースラインより高精度かつ高速・省メモリを実現。
著者: Claus Smitt, Michael Halstead, Patrick Zimmer, Thomas Läbe, Esra Guclu, Cyrill Stachniss, Chris McCool
分類: cs.RO, cs.CV, cs.LG
原文アブストラクト
Precise scene understanding is key for most robot monitoring and intervention tasks in agriculture. In this work we present PAg-NeRF which is a novel NeRF-based system that enables 3D panoptic scene understanding. Our representation is trained using an image sequence with noisy robot odometry poses and automatic panoptic predictions with inconsistent IDs between frames. Despite this noisy input, our system is able to output scene geometry, photo-realistic renders and 3D consistent panoptic representations with consistent instance IDs. We evaluate this novel system in a very challenging horticultural scenario and in doing so demonstrate an end-to-end trainable system that can make use of noisy robot poses rather than precise poses that have to be pre-calculated. Compared to a baseline approach the peak signal to noise ratio is improved from 21.34dB to 23.37dB while the panoptic quality improves from 56.65% to 70.08%. Furthermore, our approach is faster and can be tuned to improve inference time by more than a factor of 2 while being memory efficient with approximately 12 times fewer parameters.
