Advancing In-hand Dexterous Manipulation via Machine Learning

Robots are becoming better at navigating and moving around, but they still struggle with using tools, which severely limits their usefulness for household tasks. Using tools requires dexterously manipulating everyday objects like hammers, scissors, knives, screwdrivers, etc. While simple for humans, manipulating everyday objects remains a long-standing challenge that requires breakthroughs in robotic hardware, sensing, perception, and control algorithms. This thesis proposes machine learning techniques that substantially improve the state-ofthe-art performance of dexterous manipulation controllers. It focuses specifically on in-hand object reorientation tasks. Previous works on this problem had limitations like using expensive sensors or hands, only working for a few objects, requiring the hand to face upward, slow object motion, etc. This thesis goes a step further by enabling a low-cost robot hand to dynamically reorient diverse objects in mid-air with the hand facing downward using an inexpensive depth camera. To train such a system, the thesis proposes techniques for robots to learn to reorient objects with a downward-facing hand in the air. It also proposes multiple techniques to improve the time efficiency of the learning algorithms. Additionally, it discusses how to reduce the gap between simulation and reality so that controllers trained in simulation can transfer directly to real systems. Furthermore, the thesis explores the use of tactile sensors in dexterous manipulation. It concludes with a discussion of the current system’s issues and outlines future research directions for dexterous manipulation.

Name thesis:degree_name

Doctoral

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2024

Author dc:creator

• Chen, Tao

Advisor dc:contributor.advisor

• Agrawal, Pulkit

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright retained by author(s)

Licence dc:rights.uri

• https://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/156350

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/156350