{"id":{"repo_id":"mit","oai_identifier":"oai:dspace.mit.edu:1721.1/121810"},"canonical_url":"https://search.dev.ndltd.org/etd/mit/oai:dspace.mit.edu:1721.1/121810","repository":{"repo_id":"mit","name":"MIT","base_url":"https://dspace.mit.edu/oai/request"},"display":{"title":"Structured learning and inference with neural networks and generative models","abstract":"Neural networks and probabilistic models have different and in many ways complementary strengths and weaknesses: neural networks are flexible and support efficient inference, but rely on large quantities of labeled training data. Probabilistic models can learn from fewer examples, but in many cases remain limited by time-consuming inference algorithms. Thus, both classes of models have drawbacks that both limit their engineering applications and prevent them from being fully satisfying as process models of human learning. This thesis aims to address this state of affairs from both directions, exploring case studies where we make neural networks that learn from less data, and in which we design more efficient inference procedures for generative models. First, we explore recurrent neural networks that learn list-processing procedures (sort, reverse, etc.), and show how ideas from type theory and programming language theory can be used to design a data augmentation scheme that enables effective learning from small datasets. Next, we show how error-driven proposal mechanisms can speed up stochastic search for generative model inversion, first developing a symbolic model for inferring Boolean functions and Horn clause theories, and then a general-purpose neural network model for doing inference in continuous domains such as inverse graphics.","abstract_html":"Neural networks and probabilistic models have different and in many ways complementary strengths and weaknesses: neural networks are flexible and support efficient inference, but rely on large quantities of labeled training data. Probabilistic models can learn from fewer examples, but in many cases remain limited by time-consuming inference algorithms. Thus, both classes of models have drawbacks that both limit their engineering applications and prevent them from being fully satisfying as process models of human learning. This thesis aims to address this state of affairs from both directions, exploring case studies where we make neural networks that learn from less data, and in which we design more efficient inference procedures for generative models. First, we explore recurrent neural networks that learn list-processing procedures (sort, reverse, etc.), and show how ideas from type theory and programming language theory can be used to design a data augmentation scheme that enables effective learning from small datasets. Next, we show how error-driven proposal mechanisms can speed up stochastic search for generative model inversion, first developing a symbolic model for inferring Boolean functions and Horn clause theories, and then a general-purpose neural network model for doing inference in continuous domains such as inverse graphics.","abstract_has_math":false,"creators":["Lewis, Owen,Ph. D.Massachusetts Institute of Technology."],"institution":"Massachusetts Institute of Technology","degree_name":"Doctoral","degree_level":null,"degree_discipline":null,"degree_department":"Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences","school":null,"contributors":[],"advisors":["Tomaso Poggio."],"committee_chairs":[],"committee_members":[],"year":2019,"date_issued":"2019","date_published":"2019","updated_at":"2026-07-22T22:20:54Z","subjects":["Brain and Cognitive Sciences."],"languages":["eng"],"rights":["MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission."],"rights_urls":["http://dspace.mit.edu/handle/1721.1/7582"],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/1721.1/121810","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Tomaso Poggio."]},{"key":"dc:contributor.department","label":"Department","values":["Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences","Brain"]},{"key":"dc:contributor.other","label":"Dc Contributor Other","values":["Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences."]},{"key":"dc:creator","label":"Author","values":["Lewis, Owen,Ph. D.Massachusetts Institute of Technology."]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2019-07-18T20:31:33Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2019-07-18T20:31:33Z"]},{"key":"dc:date.issued","label":"Date","values":["2019"]},{"key":"dc:publisher","label":"Institution","values":["Massachusetts Institute of Technology"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctoral"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Brain and Cognitive Sciences."]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["eng"]},{"key":"dc:rights","label":"Dc Rights","values":["MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission."]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://dspace.mit.edu/handle/1721.1/7582"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/1721.1/121810"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Thesis: Ph. D., Massachusetts Institute of Technology, Department of Brain and Cognitive Sciences, 2019","Cataloged from PDF version of thesis.","Includes bibliographical references (pages 91-100)."]},{"key":"dc:description.abstract","label":"Abstract","values":["Neural networks and probabilistic models have different and in many ways complementary strengths and weaknesses: neural networks are flexible and support efficient inference, but rely on large quantities of labeled training data. Probabilistic models can learn from fewer examples, but in many cases remain limited by time-consuming inference algorithms. Thus, both classes of models have drawbacks that both limit their engineering applications and prevent them from being fully satisfying as process models of human learning. This thesis aims to address this state of affairs from both directions, exploring case studies where we make neural networks that learn from less data, and in which we design more efficient inference procedures for generative models. First, we explore recurrent neural networks that learn list-processing procedures (sort, reverse, etc.), and show how ideas from type theory and programming language theory can be used to design a data augmentation scheme that enables effective learning from small datasets. Next, we show how error-driven proposal mechanisms can speed up stochastic search for generative model inversion, first developing a symbolic model for inferring Boolean functions and Horn clause theories, and then a general-purpose neural network model for doing inference in continuous domains such as inverse graphics."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["Ph. D."]},{"key":"dc:title","label":"Title","values":["Structured learning and inference with neural networks and generative models"]}]}],"canonical_facts":{"dc:contributor.advisor":["Tomaso Poggio."],"dc:contributor.department":["Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences","Brain"],"dc:contributor.other":["Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences."],"dc:creator":["Lewis, Owen,Ph. D.Massachusetts Institute of Technology."],"dc:date.accessioned":["2019-07-18T20:31:33Z"],"dc:date.available":["2019-07-18T20:31:33Z"],"dc:date.issued":["2019"],"dc:description":["Thesis: Ph. D., Massachusetts Institute of Technology, Department of Brain and Cognitive Sciences, 2019","Cataloged from PDF version of thesis.","Includes bibliographical references (pages 91-100)."],"dc:description.abstract":["Neural networks and probabilistic models have different and in many ways complementary strengths and weaknesses: neural networks are flexible and support efficient inference, but rely on large quantities of labeled training data. Probabilistic models can learn from fewer examples, but in many cases remain limited by time-consuming inference algorithms. Thus, both classes of models have drawbacks that both limit their engineering applications and prevent them from being fully satisfying as process models of human learning. This thesis aims to address this state of affairs from both directions, exploring case studies where we make neural networks that learn from less data, and in which we design more efficient inference procedures for generative models. First, we explore recurrent neural networks that learn list-processing procedures (sort, reverse, etc.), and show how ideas from type theory and programming language theory can be used to design a data augmentation scheme that enables effective learning from small datasets. Next, we show how error-driven proposal mechanisms can speed up stochastic search for generative model inversion, first developing a symbolic model for inferring Boolean functions and Horn clause theories, and then a general-purpose neural network model for doing inference in continuous domains such as inverse graphics."],"dc:description.degree":["Ph. D."],"dc:identifier.uri":["https://hdl.handle.net/1721.1/121810"],"dc:language.iso":["eng"],"dc:publisher":["Massachusetts Institute of Technology"],"dc:rights":["MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission."],"dc:rights.uri":["http://dspace.mit.edu/handle/1721.1/7582"],"dc:subject":["Brain and Cognitive Sciences."],"dc:title":["Structured learning and inference with neural networks and generative models"],"dc:type":["Thesis"],"thesis:degree_name":["Doctoral"]},"updated_at":"2026-07-22T22:20:54Z"}