{"id":{"repo_id":"mit","oai_identifier":"oai:dspace.mit.edu:1721.1/143349"},"canonical_url":"https://search.dev.ndltd.org/etd/mit/oai:dspace.mit.edu:1721.1/143349","repository":{"repo_id":"mit","name":"MIT","base_url":"https://dspace.mit.edu/oai/request"},"display":{"title":"Meta-learning and Enforcing Useful Conservation Laws in Sequential Prediction Problems","abstract":"In recent years, deep learning techniques have enjoyed storied success in a wide array of problem domains, including computer vision, natural language processing, and robotics. While much of this success can be attributed to increasing availability of both data and computing resources, the inductive biases induced by various training methods, model components, and architectures has helped enable efficient generalization as well. Useful biases often exploit symmetries in the prediction problem, such as convolutional neural networks relying on translation equivariance. Automatically discovering such useful symmetries is a promising path to greatly improving the performance of ML systems, but it still remains a challenge. In this work, we focus on sequential prediction problems in real and simulated physical domains and take inspiration from Noether’s theorem to reduce the problem of finding inductive biases to that of meta-learning useful conserved quantities. We propose Noether Networks: a class of models where an unsupervised, meta-learned conservation loss is optimized inside the prediction function. This adapts the model weights to the particular input and imposes the approximate meta-learned conservation law in the predictions. We show, theoretically and experimentally, that Noether Networks improve prediction quality, providing a general framework for discovering inductive biases in sequential prediction problems.","abstract_html":"In recent years, deep learning techniques have enjoyed storied success in a wide array of problem domains, including computer vision, natural language processing, and robotics. While much of this success can be attributed to increasing availability of both data and computing resources, the inductive biases induced by various training methods, model components, and architectures has helped enable efficient generalization as well. Useful biases often exploit symmetries in the prediction problem, such as convolutional neural networks relying on translation equivariance. Automatically discovering such useful symmetries is a promising path to greatly improving the performance of ML systems, but it still remains a challenge. In this work, we focus on sequential prediction problems in real and simulated physical domains and take inspiration from Noether’s theorem to reduce the problem of finding inductive biases to that of meta-learning useful conserved quantities. We propose Noether Networks: a class of models where an unsupervised, meta-learned conservation loss is optimized inside the prediction function. This adapts the model weights to the particular input and imposes the approximate meta-learned conservation law in the predictions. We show, theoretically and experimentally, that Noether Networks improve prediction quality, providing a general framework for discovering inductive biases in sequential prediction problems.","abstract_has_math":false,"creators":["Doblar, Dylan D."],"institution":"Massachusetts Institute of Technology","degree_name":"Master","degree_level":null,"degree_discipline":null,"degree_department":"Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science","school":null,"contributors":[],"advisors":["Kaelbling, Leslie P."],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-02","date_published":"2022-02","updated_at":"2026-07-22T22:21:07Z","subjects":[],"languages":[],"rights":["In Copyright - Educational Use Permitted","Copyright MIT"],"rights_urls":["http://rightsstatements.org/page/InC-EDU/1.0/"],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/1721.1/143349","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Kaelbling, Leslie P."]},{"key":"dc:contributor.department","label":"Department","values":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science"]},{"key":"dc:creator","label":"Author","values":["Doblar, Dylan D."]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2022-06-15T13:14:21Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2022-06-15T13:14:21Z"]},{"key":"dc:date.issued","label":"Date","values":["2022-02"]},{"key":"dc:publisher","label":"Institution","values":["Massachusetts Institute of Technology"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master","Master of Engineering in Electrical Engineering and Computer Science"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights","label":"Dc Rights","values":["In Copyright - Educational Use Permitted","Copyright MIT"]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://rightsstatements.org/page/InC-EDU/1.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/1721.1/143349"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["In recent years, deep learning techniques have enjoyed storied success in a wide array of problem domains, including computer vision, natural language processing, and robotics. While much of this success can be attributed to increasing availability of both data and computing resources, the inductive biases induced by various training methods, model components, and architectures has helped enable efficient generalization as well. Useful biases often exploit symmetries in the prediction problem, such as convolutional neural networks relying on translation equivariance. Automatically discovering such useful symmetries is a promising path to greatly improving the performance of ML systems, but it still remains a challenge. In this work, we focus on sequential prediction problems in real and simulated physical domains and take inspiration from Noether’s theorem to reduce the problem of finding inductive biases to that of meta-learning useful conserved quantities. We propose Noether Networks: a class of models where an unsupervised, meta-learned conservation loss is optimized inside the prediction function. This adapts the model weights to the particular input and imposes the approximate meta-learned conservation law in the predictions. We show, theoretically and experimentally, that Noether Networks improve prediction quality, providing a general framework for discovering inductive biases in sequential prediction problems."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["M.Eng."]},{"key":"dc:title","label":"Title","values":["Meta-learning and Enforcing Useful Conservation Laws in Sequential Prediction Problems"]}]}],"canonical_facts":{"dc:contributor.advisor":["Kaelbling, Leslie P."],"dc:contributor.department":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science"],"dc:creator":["Doblar, Dylan D."],"dc:date.accessioned":["2022-06-15T13:14:21Z"],"dc:date.available":["2022-06-15T13:14:21Z"],"dc:date.issued":["2022-02"],"dc:description.abstract":["In recent years, deep learning techniques have enjoyed storied success in a wide array of problem domains, including computer vision, natural language processing, and robotics. While much of this success can be attributed to increasing availability of both data and computing resources, the inductive biases induced by various training methods, model components, and architectures has helped enable efficient generalization as well. Useful biases often exploit symmetries in the prediction problem, such as convolutional neural networks relying on translation equivariance. Automatically discovering such useful symmetries is a promising path to greatly improving the performance of ML systems, but it still remains a challenge. In this work, we focus on sequential prediction problems in real and simulated physical domains and take inspiration from Noether’s theorem to reduce the problem of finding inductive biases to that of meta-learning useful conserved quantities. We propose Noether Networks: a class of models where an unsupervised, meta-learned conservation loss is optimized inside the prediction function. This adapts the model weights to the particular input and imposes the approximate meta-learned conservation law in the predictions. We show, theoretically and experimentally, that Noether Networks improve prediction quality, providing a general framework for discovering inductive biases in sequential prediction problems."],"dc:description.degree":["M.Eng."],"dc:identifier.uri":["https://hdl.handle.net/1721.1/143349"],"dc:publisher":["Massachusetts Institute of Technology"],"dc:rights":["In Copyright - Educational Use Permitted","Copyright MIT"],"dc:rights.uri":["http://rightsstatements.org/page/InC-EDU/1.0/"],"dc:title":["Meta-learning and Enforcing Useful Conservation Laws in Sequential Prediction Problems"],"dc:type":["Thesis"],"thesis:degree_name":["Master","Master of Engineering in Electrical Engineering and Computer Science"]},"updated_at":"2026-07-22T22:21:07Z"}