We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
A tree-based approach for visible and thermal sensor fusion in winter autonomous driving.
- Authors
Boisclair, Jonathan; Amamou, Ali; Kelouwani, Sousso; Alam, M. Zeshan; Oueslati, Hedi; Zeghmi, Lotfi; Agbossou, Kodjo
- Abstract
Research on autonomous vehicles has been at a peak recently. One of the most researched aspects is the performance degradation of sensors in harsh weather conditions such as rain, snow, fog, and hail. This work addresses this performance degradation by fusing multiple sensor modalities inside the neural network used for detection. The proposed fusion method removes the pre-process fusion stage. It directly produces detection boxes from numerous images. It reduces the computation cost by providing detection and fusion simultaneously. By separating the network during the initial layers, the network can easily be modified for new sensors. Intra-network fusion improves robustness to missing inputs and applies to all compatible types of inputs while reducing the peak computing cost by using a valley-fill algorithm. Our experiments demonstrate that adopting a parallel multimodal network to fuse thermal images in the network improves object detection during difficult weather conditions such as harsh winters by up to 5% mAP while reducing dataset bias during complicated weather conditions. It also happens with around 50% fewer parameters than late-fusion approaches, which duplicate the whole network instead of the first section of the feature extractor.
- Subjects
TRAFFIC safety; AUTONOMOUS vehicles; WEATHER; DETECTORS; THERMOGRAPHY
- Publication
Machine Vision & Applications, 2024, Vol 35, Issue 4, p1
- ISSN
0932-8092
- Publication type
Article
- DOI
10.1007/s00138-024-01546-y