State estimation for autonomous mobile systems: Multi-IMU self-calibration and localization using 3D Gaussian splatting maps

The autonomous operation of modern autonomous mobile systems requires accurate state estimation from sensors to determine the agent's state, its surrounding environment, or both. For such sensor-based tasks to succeed, it is crucial to properly estimate and correct the parameters modeling the discrepancy between expected and actual sensor measurements. When using multiple sensors, establishing the relative pose between them—a task referred to as extrinsic calibration—is particularly important. Similarly, a primary function of sensors on autonomous systems is to establish the agent's pose within a predefined coordinate frame, a task referred to as localization. These two tasks share a common foundation, as both are fundamentally pose estimation problems typically formulated and solved through nonlinear optimization. This dissertation tackles two fundamental and complementary challenges in state estimation for autonomous systems: extrinsic calibration and localization. The first research thrust addresses the extrinsic calibration of multiple inertial measurement units (IMUs). I develop a method for multi-IMU extrinsic self-calibration that requires neither aiding sensors (e.g., cameras) nor prescribed trajectories, while relaxing key limitations of existing self-calibration methods. I then extend this method to be more efficient in terms of runtime and memory usage without compromising estimation accuracy. This is achieved by selecting and utilizing an informative subset of measurements, which is especially useful for large datasets collected over long durations where informative data is sparse. The second research thrust addresses visual localization on 3D Gaussian Splatting (3DGS) as a map representation. I develop a visual localization method based on establishing feature point correspondences between a query image and a synthetic image rendered from an initial pose estimate. This method yields significantly faster inference times and improved accuracy compared to baseline methods that iteratively minimize the photometric error between the query and rendered images. I then extend this method to a cross-modal setting, enabling the localization of an infrared query image on a 3DGS map constructed from visible-light images. This is achieved by establishing feature point correspondences between a query infrared image and a synthetic visible image rendered from an initial pose estimate using a learning-based cross-modal feature matcher, which is particularly useful for navigation under poor lighting conditions where cross-modality localization is required. All code and datasets for the developed methods are made publicly available. Collectively, this work pushes the boundaries of state estimation for autonomous mobile systems equipped with sensors, contributing to more reliable and capable autonomy in applications such as robotics, autonomous driving, unmanned flight, and extended reality.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Aerospace Engineering

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Lee, Jongwon

Contributors dc:contributor

• Bretl, Timothy
• Dullerud, Geir
• Tran, Huy
• Stefanescu, Ramona

• robotics
• state estimation
• optimization
• computer vision

Statement dc:rights

• Copyright 2025 Jongwon Lee

Language dc:language

en

Handle dc:identifier

https://hdl.handle.net/2142/132526

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/132526