Generalizable Robot Manipulation through Task and Motion Planning and Interactive Perception

For a robot operating in a daily household environment, generality is of great importance. It should be able to generalize to different tasks that involve different objects in varying backgrounds and configurations. In this thesis, we will move towards this goal from two perspectives. We will first present a strategy for designing a robot manipulation system that can generalize to a wide range of goals, environments, and objects. Such generality is achieved through task and motion planning with affordances estimated by both learned and engineered modules. We demonstrate that this strategy can enable a single policy to perform a wide variety of real-world manipulation tasks. Next, we will present an interactive perception solution to deal with the uncertainty in the estimated affordances, with a focus on the segmentation of objects. We adopt an object-based belief representation to estimate the uncertainty coming from predicted segmentation, and select actions to reduce that efficiently. Our experiments show that our system can generalize better to different environments and reduce uncertainty more efficiently compared to our baselines.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2022

Author dc:creator

• Fang, Xiaolin

Advisors dc:contributor.advisor

• Kaelbling, Leslie Pack
• Lozano-Pérez, Tomás

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright MIT

Licence dc:rights.uri

• http://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/144945

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/144945