AgriNeRF: 厳しい照明条件下での農業向けニューラル放射場
AgriNeRF: Neural Radiance Fields for Agriculture in Challenging Lighting Conditions
RGB・イベント・熱カメラを組み合わせたNeRFで、低照度や極端な明るさなど厳しい照明下でも農業シーンの3D再構成と果実検出を高精度化した研究。
著者: Samarth Chopra, Fernando Cladera, Varun Murali, Vijay Kumar
分類: cs.RO
原文アブストラクト
Neural Radiance Fields (NeRFs) have shown significant promise in 3D scene reconstruction and novel view synthesis. In agricultural settings, NeRFs can serve as digital twins, providing critical information about fruit detection for yield estimation and other important metrics for farmers. However, traditional NeRFs are not robust to challenging lighting conditions, such as low-light, extreme bright light and varying lighting. To address these issues, this work leverages three different sensors: an RGB camera, an event camera and a thermal camera. Our RGB scene reconstruction shows an improvement in PSNR and SSIM by +2.06 dB and +8.3% respectively. Our cross-spectral scene reconstruction enhances downstream fruit detection by +43.0% in mAP50 and +61.1% increase in mAP50-95. The integration of additional sensors leads to a more robust and informative NeRF. We demonstrate that our multi-modal system yields high quality photo-realistic reconstructions under various tree canopy covers and at different times of the day. This work results in the development of a resilient NeRF, capable of performing well in visibly degraded scenarios, as well as a learnt cross-spectral representation, that is used for automated fruit detection.
