Introspective learning based Visual-LiDAR fusion for adaptive Simultaneous Localization and Mapping

This work is about developing an adaptive Visual-LiDAR Simultaneous Localization and Mapping (SLAM) algorithm. The objective is to develop a SLAM system that can adaptively negotiate LiDAR degenerate scenarios and visually challenging environments using sensor fusion. The fusion is based on the Pose Graph Optimization (PGO) technique, utilizing adaptive fusion weights predicted from a Deep Neural Network (DNN) model. The DNN model framework is inspired by introspective learning for vision systems. The DNN model is trained on a large dataset called TartanAir, which has diverse and challenging environmental conditions. The output of the model is the predicted error on the visual odometry and LiDAR odometry, which is used to compose the information matrix of the PGO. The PGO framework with weighted pose constraints from visual odometry, LiDAR odometry, and loop closure is solved using the Levenberg–Marquardt optimization algorithm. The proposed framework shows superior performance compared to the visual-only, LiDAR-only SLAM, and baseline fusion methods that were evaluated in this study.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Mechanical Engineering

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2023

Author dc:creator

• Kedia, Shubham

Contributors dc:contributor

• Hauser, Kris

• Slam
• Computer Vision
• Sensor Fusion
• State Estimation
• Deep Learning
• Robotics
• Optimization
• Adaptive Slam

Statement dc:rights

• Copyright 2023 Shubham Kedia

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/121406

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F121406