Dynamic and structured scene representation for robotic manipulation

Representation is an essential component of typical robotic manipulation frameworks. An ideal representation should be both efficient for computation and sufficient for downstream tasks. However, existing representations typically have fixed dimensions, which may not be optimal for different tasks. Therefore, in the first part of the thesis, we propose a dynamic representation that can dynamically adapt its dimensions to current the observation and the goal. Through various pile manipulation experiments, we demonstrate that dynamic representation can significantly improve the performance of robotic manipulation tasks compared to fixed- size representations. In the second part of the thesis, we propose another novel implicit representation, D³Fields, that is 3D, semantic, and dynamic. Such a representation can be used for zero-shot generalizable rearrangement tasks, where the goal is specified by 2D images. We demonstrate the effectiveness of our D³Fields through a wide range of robotic rearrangement tasks, including organizing shoes, collecting debris, and organizing office desks. Compared to state-of-the-art implicit 3D representations, such as FeatureNeRF and DistilledNeRF [1], [2], our D³Fields is more computationally efficient and effective for novel scenes.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Wang, Yixuan

Contributors dc:contributor

• Li, Yunzhu
• Driggs-Campbell, Katie
• Hajek, Bruce

• Robotic Manipulation
• Robot Learning
• Representation Learning

Statement dc:rights

• Copyright 2024 Yixuan Wang

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/125573

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F125573