{"id":{"repo_id":"mit","oai_identifier":"oai:dspace.mit.edu:1721.1/127352"},"canonical_url":"https://search.dev.ndltd.org/etd/mit/oai:dspace.mit.edu:1721.1/127352","repository":{"repo_id":"mit","name":"MIT","base_url":"https://dspace.mit.edu/oai/request"},"display":{"title":"Learning compositional dynamics models for model-based control","abstract":"Compared with off-the-shelf physics engines, a learnable simulator has a stronger ability to adapt to unseen objects, scenes, and tasks. However, existing models like Interaction Networks only work for fully observable systems; they also only consider pairwise interactions within a single time step, both restricting their use in practical systems. We introduce Propagation Networks (PropNets), a differentiable, learnable dynamics model that handles partially observable scenarios and enables instantaneous propagation of signals beyond pairwise interactions. In the second half of the thesis, I will discuss our attempt to extend PropNets to learn a particle-based simulator for handling matters of various substances--rigid or soft bodies, liquid, gas--each with distinct physical behaviors. Combining learning with particle-based systems brings in two major benefits: first, the learned simulator, just like other particle-based systems, acts widely on objects of different materials; second, the particle-based representation poses strong inductive bias for learning: particles of the same type have the same dynamics within. We demonstrate that our models not only outperform current learnable physics engines in forward simulation, but also achieve superior performance on various control tasks, such as manipulating a pile of boxes, a cup of water, and a deformable foam, with experiments both in simulation and in the real world. Compared with existing model-free deep reinforcement learning algorithms, model-based control with our models is also more accurate, efficient, and generalizable to new, partially observable scenes and tasks.","abstract_html":"Compared with off-the-shelf physics engines, a learnable simulator has a stronger ability to adapt to unseen objects, scenes, and tasks. However, existing models like Interaction Networks only work for fully observable systems; they also only consider pairwise interactions within a single time step, both restricting their use in practical systems. We introduce Propagation Networks (PropNets), a differentiable, learnable dynamics model that handles partially observable scenarios and enables instantaneous propagation of signals beyond pairwise interactions. In the second half of the thesis, I will discuss our attempt to extend PropNets to learn a particle-based simulator for handling matters of various substances--rigid or soft bodies, liquid, gas--each with distinct physical behaviors. Combining learning with particle-based systems brings in two major benefits: first, the learned simulator, just like other particle-based systems, acts widely on objects of different materials; second, the particle-based representation poses strong inductive bias for learning: particles of the same type have the same dynamics within. We demonstrate that our models not only outperform current learnable physics engines in forward simulation, but also achieve superior performance on various control tasks, such as manipulating a pile of boxes, a cup of water, and a deformable foam, with experiments both in simulation and in the real world. Compared with existing model-free deep reinforcement learning algorithms, model-based control with our models is also more accurate, efficient, and generalizable to new, partially observable scenes and tasks.","abstract_has_math":false,"creators":["Li, Yunzhu(Scientist in electrical engineering and computer science)Massachusetts Institute of Technology."],"institution":"Massachusetts Institute of Technology","degree_name":"Master","degree_level":null,"degree_discipline":null,"degree_department":"Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science","school":null,"contributors":[],"advisors":["Antonio Torralba and Russ Tedrake."],"committee_chairs":[],"committee_members":[],"year":2020,"date_issued":"2020","date_published":"2020","updated_at":"2026-07-22T22:22:04Z","subjects":["Electrical Engineering and Computer Science."],"languages":["eng"],"rights":["MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided."],"rights_urls":["http://dspace.mit.edu/handle/1721.1/7582"],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/1721.1/127352","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Antonio Torralba and Russ Tedrake."]},{"key":"dc:contributor.department","label":"Department","values":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science","EECS"]},{"key":"dc:contributor.other","label":"Dc Contributor Other","values":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science."]},{"key":"dc:creator","label":"Author","values":["Li, Yunzhu(Scientist in electrical engineering and computer science)Massachusetts Institute of Technology."]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2020-09-15T21:53:33Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2020-09-15T21:53:33Z"]},{"key":"dc:date.issued","label":"Date","values":["2020"]},{"key":"dc:publisher","label":"Institution","values":["Massachusetts Institute of Technology"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Electrical Engineering and Computer Science."]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["eng"]},{"key":"dc:rights","label":"Dc Rights","values":["MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided."]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://dspace.mit.edu/handle/1721.1/7582"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/1721.1/127352"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, May, 2020","Cataloged from the official PDF of thesis.","Includes bibliographical references (pages 55-58)."]},{"key":"dc:description.abstract","label":"Abstract","values":["Compared with off-the-shelf physics engines, a learnable simulator has a stronger ability to adapt to unseen objects, scenes, and tasks. However, existing models like Interaction Networks only work for fully observable systems; they also only consider pairwise interactions within a single time step, both restricting their use in practical systems. We introduce Propagation Networks (PropNets), a differentiable, learnable dynamics model that handles partially observable scenarios and enables instantaneous propagation of signals beyond pairwise interactions. In the second half of the thesis, I will discuss our attempt to extend PropNets to learn a particle-based simulator for handling matters of various substances--rigid or soft bodies, liquid, gas--each with distinct physical behaviors. Combining learning with particle-based systems brings in two major benefits: first, the learned simulator, just like other particle-based systems, acts widely on objects of different materials; second, the particle-based representation poses strong inductive bias for learning: particles of the same type have the same dynamics within. We demonstrate that our models not only outperform current learnable physics engines in forward simulation, but also achieve superior performance on various control tasks, such as manipulating a pile of boxes, a cup of water, and a deformable foam, with experiments both in simulation and in the real world. Compared with existing model-free deep reinforcement learning algorithms, model-based control with our models is also more accurate, efficient, and generalizable to new, partially observable scenes and tasks."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["S.M."]},{"key":"dc:title","label":"Title","values":["Learning compositional dynamics models for model-based control"]}]}],"canonical_facts":{"dc:contributor.advisor":["Antonio Torralba and Russ Tedrake."],"dc:contributor.department":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science","EECS"],"dc:contributor.other":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science."],"dc:creator":["Li, Yunzhu(Scientist in electrical engineering and computer science)Massachusetts Institute of Technology."],"dc:date.accessioned":["2020-09-15T21:53:33Z"],"dc:date.available":["2020-09-15T21:53:33Z"],"dc:date.issued":["2020"],"dc:description":["Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, May, 2020","Cataloged from the official PDF of thesis.","Includes bibliographical references (pages 55-58)."],"dc:description.abstract":["Compared with off-the-shelf physics engines, a learnable simulator has a stronger ability to adapt to unseen objects, scenes, and tasks. However, existing models like Interaction Networks only work for fully observable systems; they also only consider pairwise interactions within a single time step, both restricting their use in practical systems. We introduce Propagation Networks (PropNets), a differentiable, learnable dynamics model that handles partially observable scenarios and enables instantaneous propagation of signals beyond pairwise interactions. In the second half of the thesis, I will discuss our attempt to extend PropNets to learn a particle-based simulator for handling matters of various substances--rigid or soft bodies, liquid, gas--each with distinct physical behaviors. Combining learning with particle-based systems brings in two major benefits: first, the learned simulator, just like other particle-based systems, acts widely on objects of different materials; second, the particle-based representation poses strong inductive bias for learning: particles of the same type have the same dynamics within. We demonstrate that our models not only outperform current learnable physics engines in forward simulation, but also achieve superior performance on various control tasks, such as manipulating a pile of boxes, a cup of water, and a deformable foam, with experiments both in simulation and in the real world. Compared with existing model-free deep reinforcement learning algorithms, model-based control with our models is also more accurate, efficient, and generalizable to new, partially observable scenes and tasks."],"dc:description.degree":["S.M."],"dc:identifier.uri":["https://hdl.handle.net/1721.1/127352"],"dc:language.iso":["eng"],"dc:publisher":["Massachusetts Institute of Technology"],"dc:rights":["MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided."],"dc:rights.uri":["http://dspace.mit.edu/handle/1721.1/7582"],"dc:subject":["Electrical Engineering and Computer Science."],"dc:title":["Learning compositional dynamics models for model-based control"],"dc:type":["Thesis"],"thesis:degree_name":["Master"]},"updated_at":"2026-07-22T22:22:04Z"}