Representing and manipulating deformable linear objects

This dissertation presents innovations in three areas necessary for robots interacting with Deformable Linear Objects (DLOs): detection, tracking, and manipulation. These skills are essential in tasks such as surgical suturing and cable management in engineering systems. Unlike rigid bodies with fixed geometry and well-studied dynamics, DLOs undergo continuous deformation; have many degrees of freedom; have difficult-to-measure material properties; and frequently experience self-occlusions, self-collisions, environmental occlusions, and environmental collisions. The complex visual and physics characteristics of DLOs complicate their modeling and control. This work distinguishes itself by presenting integrated end-to-end systems for perceiving and manipulating DLOs using cameras and robots, while much of the related work relies on methods developed or validated in simulators. Although this work does not solve the full complexity of suturing or cable management, it tackles fundamental perception and manipulation challenges that arise in these demanding applications, using real robot hardware to make progress toward more capable DLO autonomy. Detecting - COCOpen is an image dataset for instance segmentation of scenes containing both rigid objects and Ethernet cables. It features an augmentation approach that grounds synthetic images in a real world context by automatically labeling wire images that classic computer vision techniques can annotate reliably and combining them with varied background scenes. This strategy provides scalable training data for training Ethernet cable detectors at low acquisition and annotation cost. Tracking - MultiDLO is a real-time algorithm for simultaneously estimating the 3D shapes of multiple entangling DLOs from RGB-D videos, which had previously never been done. It initializes by applying instance segmentation on the first frame, then requires only background segmentation for the remainder of the sequence. TrackDLO is a real-time algorithm for estimating the 3D shape of a single DLO from RGB-D videos during challenging occlusion scenarios: tip occlusion, mid-section occlusion, and self-occlusion. It is the first method to achieve accurate tracking without privileged input from fiducial markers, physics-based simulation, or robot trajectories. DLO-Splatting is an algorithm for estimating the 3D shape of a single DLO from RGB-D videos and a gripper state for tracking DLO states through self-collisions. Manipulating - KnotDLO is a method for one-handed knot tying that is repeatable for varying rope initial planar configurations and requires no human demonstrations or training. Grasp and target waypoints for future DLO states are planned from the current DLO shape. GraphDLO is a method for direct trajectory prediction of a rope that computes the directory only from the current rope state and a planned gripper action sequence. It also includes the first large-scale, real-world labeled rope interaction dataset, collected autonomously over 300 hours using a cloud robotics platform. All code and data for COCOpen, MultiDLO, TrackDLO, DLO-Splatting, KnotDLO, and GraphDLO is publicly available to support future benchmarking in detection, tracking, and manipulation of DLOs. This work helps robots see, follow, and use DLOs, moving toward using them as tools for transforming the world.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Aerospace Engineering

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Dinkel, Holly M.

Contributors dc:contributor

• Bretl, Timothy
• Coltin, Brian
• Smith, Trey
• Amato, Nancy
• Tran, Huy
• Yim, Justin

• Robotics
• Computer Vision
• Robotic Perception
• Perception
• Robotic Manipulation
• Manipulation
• Object Detection
• Deformable Objects
• Deformable Object Detection
• Object Tracking
• Tracking
• Deformable Object Tracking
• Manipulating Deformable Objects
• Prediction
• Trajectory Prediction

Statement dc:rights

• Copyright 2025 Holly M. Dinkel

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/129943

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F129943