{"id":{"repo_id":"cornell","oai_identifier":"oai:ecommons.cornell.edu:1813/51619"},"canonical_url":"https://search.dev.ndltd.org/etd/cornell/oai:ecommons.cornell.edu:1813/51619","repository":{"repo_id":"cornell","name":"Cornell University","base_url":"https://ecommons.cornell.edu/server/oai/request"},"display":{"title":"Learning to Manipulate Novel Objects for Assistive Robots","abstract":"The ability to reason about different modalities of information, for the purpose of physical interaction with objects, is a critical skill for assistive robots. For a robot to be able to assist us in our daily lives, it is not feasible to train each robot for a large number of tasks with all instances of objects that exist in human environments. Robots will have to generalize their skills by jointly reasoning with various sensor modalities such as vision, language and haptic feedback. This is an extremely challenging problem because each modality has intrinsically different statistical properties. Moreover, even with expert knowledge, manually designing joint features between such disparate modalities is difficult. In this dissertation, we focus on developing learning algorithms for robots that model tasks involving interactions with various objects in unstructured human environments --- especially on novel objects and scenarios that involve sequences of complicated manipulation. To this end, we develop algorithms that learn shared representations of multimodal data and model full sequences of complex motions. We demonstrate our approach on several different applications: understanding human activities in unstructured environment, synthesizing manipulation sequences for under-specified tasks, manipulating novel appliances, and manipulating objects with haptic feedback.","abstract_html":"The ability to reason about different modalities of information, for the purpose of physical interaction with objects, is a critical skill for assistive robots. For a robot to be able to assist us in our daily lives, it is not feasible to train each robot for a large number of tasks with all instances of objects that exist in human environments. Robots will have to generalize their skills by jointly reasoning with various sensor modalities such as vision, language and haptic feedback. This is an extremely challenging problem because each modality has intrinsically different statistical properties. Moreover, even with expert knowledge, manually designing joint features between such disparate modalities is difficult. In this dissertation, we focus on developing learning algorithms for robots that model tasks involving interactions with various objects in unstructured human environments --- especially on novel objects and scenarios that involve sequences of complicated manipulation. To this end, we develop algorithms that learn shared representations of multimodal data and model full sequences of complex motions. We demonstrate our approach on several different applications: understanding human activities in unstructured environment, synthesizing manipulation sequences for under-specified tasks, manipulating novel appliances, and manipulating objects with haptic feedback.","abstract_has_math":false,"creators":["Sung, Jaeyong"],"institution":"Cornell University","degree_name":"Ph. D., Computer Science","degree_level":"Doctor of Philosophy","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":[],"advisors":[],"committee_chairs":[],"committee_members":["Salisbury, J. Kenneth","Selman, Bart","Guimbretière, François","Marschner, Steve"],"year":2017,"date_issued":"2017-05-30","date_published":"2017-05-30","updated_at":"2026-07-24T01:49:08Z","subjects":["machine learning","Multimodal Data","Robotic Manipulation","Robot Learning","Artificial intelligence","Deep Learning","Computer science","Robotics"],"languages":["en_US"],"rights":[],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier.doi","label":"DOI","values":["https://doi.org/10.7298/X43R0R0W"],"render_values":[{"text":"https://doi.org/10.7298/X43R0R0W","href":"https://doi.org/10.7298/X43R0R0W","code":true}]},{"key":"dc:identifier.other","label":"Dc Identifier Other","values":["ProQuest Submission ID: 10207","ProQuest Publication ID: 10258261"],"render_values":[{"text":"ProQuest Submission ID: 10207","href":null,"code":true},{"text":"ProQuest Publication ID: 10258261","href":null,"code":true}]}]},"links":{"outbound_url":"https://hdl.handle.net/1813/51619","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.committeemember","label":"Committee Member","values":["Salisbury, J. Kenneth","Selman, Bart","Guimbretière, François","Marschner, Steve"]},{"key":"dc:creator","label":"Author","values":["Sung, Jaeyong"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2017-07-07T12:48:41Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2017-12-08T07:00:46Z"]},{"key":"dc:date.issued","label":"Date","values":["2017-05-30"]},{"key":"dc:type","label":"Dc Type","values":["dissertation or thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Doctor of Philosophy"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph. D., Computer Science"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["Cornell University"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["machine learning","Multimodal Data","Robotic Manipulation","Robot Learning","Artificial intelligence","Deep Learning","Computer science","Robotics"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en_US"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.doi","label":"DOI","values":["https://doi.org/10.7298/X43R0R0W"]},{"key":"dc:identifier.other","label":"Dc Identifier Other","values":["ProQuest Submission ID: 10207","ProQuest Publication ID: 10258261"]},{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/1813/51619"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["The ability to reason about different modalities of information, for the purpose of physical interaction with objects, is a critical skill for assistive robots. For a robot to be able to assist us in our daily lives, it is not feasible to train each robot for a large number of tasks with all instances of objects that exist in human environments. Robots will have to generalize their skills by jointly reasoning with various sensor modalities such as vision, language and haptic feedback. This is an extremely challenging problem because each modality has intrinsically different statistical properties. Moreover, even with expert knowledge, manually designing joint features between such disparate modalities is difficult. In this dissertation, we focus on developing learning algorithms for robots that model tasks involving interactions with various objects in unstructured human environments --- especially on novel objects and scenarios that involve sequences of complicated manipulation. To this end, we develop algorithms that learn shared representations of multimodal data and model full sequences of complex motions. We demonstrate our approach on several different applications: understanding human activities in unstructured environment, synthesizing manipulation sequences for under-specified tasks, manipulating novel appliances, and manipulating objects with haptic feedback."]},{"key":"dc:format.mimetype","label":"Dc Format Mimetype","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Learning to Manipulate Novel Objects for Assistive Robots"]}]}],"canonical_facts":{"dc:contributor.committeemember":["Salisbury, J. Kenneth","Selman, Bart","Guimbretière, François","Marschner, Steve"],"dc:creator":["Sung, Jaeyong"],"dc:date.accessioned":["2017-07-07T12:48:41Z"],"dc:date.available":["2017-12-08T07:00:46Z"],"dc:date.issued":["2017-05-30"],"dc:description.abstract":["The ability to reason about different modalities of information, for the purpose of physical interaction with objects, is a critical skill for assistive robots. For a robot to be able to assist us in our daily lives, it is not feasible to train each robot for a large number of tasks with all instances of objects that exist in human environments. Robots will have to generalize their skills by jointly reasoning with various sensor modalities such as vision, language and haptic feedback. This is an extremely challenging problem because each modality has intrinsically different statistical properties. Moreover, even with expert knowledge, manually designing joint features between such disparate modalities is difficult. In this dissertation, we focus on developing learning algorithms for robots that model tasks involving interactions with various objects in unstructured human environments --- especially on novel objects and scenarios that involve sequences of complicated manipulation. To this end, we develop algorithms that learn shared representations of multimodal data and model full sequences of complex motions. We demonstrate our approach on several different applications: understanding human activities in unstructured environment, synthesizing manipulation sequences for under-specified tasks, manipulating novel appliances, and manipulating objects with haptic feedback."],"dc:format.mimetype":["application/pdf"],"dc:identifier.doi":["https://doi.org/10.7298/X43R0R0W"],"dc:identifier.other":["ProQuest Submission ID: 10207","ProQuest Publication ID: 10258261"],"dc:identifier.uri":["https://hdl.handle.net/1813/51619"],"dc:language.iso":["en_US"],"dc:subject":["machine learning","Multimodal Data","Robotic Manipulation","Robot Learning","Artificial intelligence","Deep Learning","Computer science","Robotics"],"dc:title":["Learning to Manipulate Novel Objects for Assistive Robots"],"dc:type":["dissertation or thesis"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Doctor of Philosophy"],"thesis:degree_name":["Ph. D., Computer Science"],"thesis:institution_name":["Cornell University"]},"updated_at":"2026-07-24T01:49:08Z"}