{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/132632"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/132632","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Vision-based proprioception and tactile sensing for soft robots","abstract":"Soft pneumatic manipulators are attractive for industrial and human-interactive tasks because of their inherent compliance, yet practical deployment demands accurate proprioception and tactile feedback. This thesis introduces a compact, vision-based sensing framework that delivers both modalities from a single internal camera. We instantiate the approach on PneuGelSight, a pneumatically actuated finger that uses color-coded illumination and a reflective elastomer surface to encode deformation and contact geometry in one image. To co-design hardware and perception, we develop a simulation pipeline that couples finite-element deformation with physics-based optical rendering, enabling design optimization and training data generation. The resulting models provide high-resolution proprioceptive shape estimation and dense tactile reconstruction, transferring from simulation to hardware without per-scene supervision (zero-shot). Experiments demonstrate accurate recovery of large-scale bends, robust contact mapping under varied loads, and practical multi-touch object reconstruction, while keeping hardware simple and lightweight. Together, PneuGelSight and the sim-to-real pipeline offer an easily implementable and robust sensing methodology for soft robots, advancing the integration of rich feedback into compliant manipulators and opening paths to closed-loop control and scalable multi-finger systems.","abstract_html":"Soft pneumatic manipulators are attractive for industrial and human-interactive tasks because of their inherent compliance, yet practical deployment demands accurate proprioception and tactile feedback. This thesis introduces a compact, vision-based sensing framework that delivers both modalities from a single internal camera. We instantiate the approach on PneuGelSight, a pneumatically actuated finger that uses color-coded illumination and a reflective elastomer surface to encode deformation and contact geometry in one image. To co-design hardware and perception, we develop a simulation pipeline that couples finite-element deformation with physics-based optical rendering, enabling design optimization and training data generation. The resulting models provide high-resolution proprioceptive shape estimation and dense tactile reconstruction, transferring from simulation to hardware without per-scene supervision (zero-shot). Experiments demonstrate accurate recovery of large-scale bends, robust contact mapping under varied loads, and practical multi-touch object reconstruction, while keeping hardware simple and lightweight. Together, PneuGelSight and the sim-to-real pipeline offer an easily implementable and robust sensing methodology for soft robots, advancing the integration of rich feedback into compliant manipulators and opening paths to closed-loop control and scalable multi-finger systems.","abstract_has_math":false,"creators":["Zhang, Ruohan"],"institution":"University of Illinois Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Yuan, Wenzhen"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-12","date_published":"2025-12","updated_at":"2026-07-22T22:25:07Z","subjects":["Tactile Sensing","Robot Perception"],"languages":["en"],"rights":["Copyright 2025 Ruohan Zhang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/132632","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Yuan, Wenzhen"]},{"key":"dc:creator","label":"Author","values":["Zhang, Ruohan"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-12","2025-12-12"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Tactile Sensing","Robot Perception"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2025 Ruohan Zhang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/132632"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Soft pneumatic manipulators are attractive for industrial and human-interactive tasks because of their inherent compliance, yet practical deployment demands accurate proprioception and tactile feedback. This thesis introduces a compact, vision-based sensing framework that delivers both modalities from a single internal camera. We instantiate the approach on PneuGelSight, a pneumatically actuated finger that uses color-coded illumination and a reflective elastomer surface to encode deformation and contact geometry in one image. To co-design hardware and perception, we develop a simulation pipeline that couples finite-element deformation with physics-based optical rendering, enabling design optimization and training data generation. The resulting models provide high-resolution proprioceptive shape estimation and dense tactile reconstruction, transferring from simulation to hardware without per-scene supervision (zero-shot). Experiments demonstrate accurate recovery of large-scale bends, robust contact mapping under varied loads, and practical multi-touch object reconstruction, while keeping hardware simple and lightweight. Together, PneuGelSight and the sim-to-real pipeline offer an easily implementable and robust sensing methodology for soft robots, advancing the integration of rich feedback into compliant manipulators and opening paths to closed-loop control and scalable multi-finger systems.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-12-01","The student, Ruohan Zhang, accepted the attached license on 2025-12-11 at 16:26.","The student, Ruohan Zhang, submitted this Thesis for approval on 2025-12-11 at 16:27.","This Thesis was approved for publication on 2025-12-12 at 08:22.","DSpace SAF Submission Ingestion Package generated from Vireo submission #22831 on 2026-02-19 at 18:45:37"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Vision-based proprioception and tactile sensing for soft robots"]}]}],"canonical_facts":{"dc:contributor":["Yuan, Wenzhen"],"dc:creator":["Zhang, Ruohan"],"dc:date":["2025-12","2025-12-12"],"dc:description":["Soft pneumatic manipulators are attractive for industrial and human-interactive tasks because of their inherent compliance, yet practical deployment demands accurate proprioception and tactile feedback. This thesis introduces a compact, vision-based sensing framework that delivers both modalities from a single internal camera. We instantiate the approach on PneuGelSight, a pneumatically actuated finger that uses color-coded illumination and a reflective elastomer surface to encode deformation and contact geometry in one image. To co-design hardware and perception, we develop a simulation pipeline that couples finite-element deformation with physics-based optical rendering, enabling design optimization and training data generation. The resulting models provide high-resolution proprioceptive shape estimation and dense tactile reconstruction, transferring from simulation to hardware without per-scene supervision (zero-shot). Experiments demonstrate accurate recovery of large-scale bends, robust contact mapping under varied loads, and practical multi-touch object reconstruction, while keeping hardware simple and lightweight. Together, PneuGelSight and the sim-to-real pipeline offer an easily implementable and robust sensing methodology for soft robots, advancing the integration of rich feedback into compliant manipulators and opening paths to closed-loop control and scalable multi-finger systems.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2027-12-01","The student, Ruohan Zhang, accepted the attached license on 2025-12-11 at 16:26.","The student, Ruohan Zhang, submitted this Thesis for approval on 2025-12-11 at 16:27.","This Thesis was approved for publication on 2025-12-12 at 08:22.","DSpace SAF Submission Ingestion Package generated from Vireo submission #22831 on 2026-02-19 at 18:45:37"],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/132632"],"dc:language":["en"],"dc:rights":["Copyright 2025 Ruohan Zhang"],"dc:subject":["Tactile Sensing","Robot Perception"],"dc:title":["Vision-based proprioception and tactile sensing for soft robots"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:07Z"}