Learning 3D Robotics Perception using Inductive Priors

Recent advances in deep learning have led to a data-centric intelligence in the last decade, i.e. artificially intelligent models unlocking the potential to ingest a large amount of data and be really good at performing digital tasks such as text-to-image generation, machine-human conversation, and image recognition. This thesis covers the topic of learning with structured inductive bias and priors to design approaches and algorithms unlocking the potential of principle-centric intelligence for the real-world. Prior knowledge (priors for short), often available in terms of past experience as well as assumptions of how the world works, helps the autonomous agent generalize better and adapt their behavior based on past experience. In this thesis, I demonstrate the use of prior knowledge in three different robotics perception problems. 1. object-centric 3D reconstruction, 2. vision and language for decision-making, and 3. 3D scene understanding. To solve these challenging problems, I propose various sources of prior knowledge including 1. geometry and appearance priors from synthetic data, 2. modularity and semantic map priors and 3. semantic, structural, and contextual priors. I study these priors for solving robotics 3D perception tasks and propose ways to efficiently encode them in deep learning models. Some priors are used to warm-start the network for transfer learning, others are used as hard constraints to restrict the action space of robotics agents. While classical techniques are brittle and fail to generalize to unseen scenarios and data-centric approaches require a large amount of labeled data, this thesis aims to build intelligent agents which require very-less real-world data or data acquired only from simulation to generalize to highly dynamic and cluttered environments in novel simulations (i.e. sim2sim) or real-world unseen environments (i.e. sim2real) for a holistic scene understanding of the 3D world.

Level thesis:degree_level

Doctoral

Department dc:contributor.department

Mechanical Engineering

Grantor dc:publisher

Georgia Institute of Technology

Year dc:date.issued

2023

Author dc:creator

• Irshad, Muhammad Zubair

Advisors dc:contributor.advisor

• Kira, Zsolt
• Young, Aaron

Committee members dc:contributor.committeemember

• Sadegh, Nader
• Kousik, Shreyas
• Gaidon, Adrien

• 3D perception
• Deep learning
• Implicit representation
• Self-supervised learning
• Neural radiance field
• 6D pose estimation
• 3D reconstruction
• Vision-and-Language Navigation

Language dc:language.iso

en_US

Handle dc:identifier.uri

https://hdl.handle.net/1853/73171

OAI identifier oai:identifier

oai:repository.gatech.edu:1853/73171