Coverage and exploration planning with semantically informed maps

Outside of controlled factory environments, robots are expected to build their own environment representations to effectively plan and act in the world. This representation typically takes the shape of a semantic map constructed through some Simultaneous Localization and Mapping pipeline. In this thesis, I first present my work on a robotic application in the wild whose performance relies not just on the accuracy of the resulting metric-semantic world map, but also on its uncertainty: Semantically Informed Ultraviolet Disinfection Planning. Taking this application as an example of semantically informed coverage planning, I then present evidence that most of the sequential estimation strategies for semantic maps lead to overconfident label estimates, and that this overconfidence impacts the downstream performance of the robotic agents, both in disinfection and object goal navigation tasks. I also provide two novel confidence-preserving memory-efficient methods to perform online metric-semantic reconstruction in real time. I then extend the domain of coverage planning problems to include the coverage of interactive scenes by proposing a solution to the Manipulation-Enhanced Mapping problem, where a robot must efficiently survey an environment with a camera and manipulate objects in the environment to improve object visibility. This is achieved by leveraging neural networks to accelerate the belief updates of well-studied formalism for decision-making under uncertainty of Partially Observable Markov Decision Processes, enabling us to solve these problems in the belief space of metric-semantic maps. I finally conclude with a summary of the presented research, as well as promising future research directions in the areas of coverage planning, interactive scene representations and on further improving confidence calibration and uncertainty handling in 3D metric-semantic maps.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Correia Marques, Joao Marcos

Contributors dc:contributor

• Hauser, Kris
• Wang, Shenlong
• Hoiem, Derek
• Atanasov, Nikolay

• Metric-semantic Reconstruction
• Semantic Mapping
• Coverage Planning
• Ultraviolet Disinfection
• Coverage Path Planning
• Manipulation-enhanced Mapping
• Confidence Calibration
• 3d Mapping

Statement dc:rights

• © 2025 Joao Marcos Correia Marques

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/129831

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F129831