Image-based pose estimation of sub-centimeter industrial parts for robotic grasping

This work integrates existing machine vision techniques with proposed refinement method to estimate the 6DoF pose of sub-centimeter parts from images for high-mix, low-volume assembly lines. In this system, the 3D models of three parts are input to a BlenderProc2 rendering engine to generate a physically- and photometrically-realistic synthetic image dataset. Synthetic images are used to train a Mask R-CNN model for segmenting individual part instances in a scene, with automatically-generated instance mask labels, eliminating the need for manual labeling. Instance segmentation enables part selection for assembly when multiple parts are present. Additionally, a PVNet model is trained on cropped images of each part instance from synthetic data to estimate their positions and orientations. An additional pose refinement step adjusts PVNet pose estimates by aligning the orientation to the nearest physically-stable configuration on a planar surface and refining the translation using calibrated object-to-camera distances from the workspace. To evaluate robustness, noise is injected into the keypoint detection stage of the PVNet model in an ablation study to assess the impact of sensor noise on pose estimation. Real robot pick-and-place experiments demonstrate the system performance.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Mechanical Engineering

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Dai, Yangfei

Contributors dc:contributor

• Bretl, Timothy

• Computer Vision
• 6dof Pose Estimation
• Synthetic Data Generation

Statement dc:rights

• Copyright 2024 Yangfei Dai

Language dc:language

eng, en

Handle dc:identifier

https://hdl.handle.net/2142/127291

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F127291