Vision-based proprioception and tactile sensing for soft robots

Soft pneumatic manipulators are attractive for industrial and human-interactive tasks because of their inherent compliance, yet practical deployment demands accurate proprioception and tactile feedback. This thesis introduces a compact, vision-based sensing framework that delivers both modalities from a single internal camera. We instantiate the approach on PneuGelSight, a pneumatically actuated finger that uses color-coded illumination and a reflective elastomer surface to encode deformation and contact geometry in one image. To co-design hardware and perception, we develop a simulation pipeline that couples finite-element deformation with physics-based optical rendering, enabling design optimization and training data generation. The resulting models provide high-resolution proprioceptive shape estimation and dense tactile reconstruction, transferring from simulation to hardware without per-scene supervision (zero-shot). Experiments demonstrate accurate recovery of large-scale bends, robust contact mapping under varied loads, and practical multi-touch object reconstruction, while keeping hardware simple and lightweight. Together, PneuGelSight and the sim-to-real pipeline offer an easily implementable and robust sensing methodology for soft robots, advancing the integration of rich feedback into compliant manipulators and opening paths to closed-loop control and scalable multi-finger systems.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Zhang, Ruohan

Contributors dc:contributor

• Yuan, Wenzhen

• Tactile Sensing
• Robot Perception

Statement dc:rights

• Copyright 2025 Ruohan Zhang

Language dc:language

en

Handle dc:identifier

https://hdl.handle.net/2142/132632

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/132632