{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/115878"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/115878","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Applications of diffusion processes: machine learning, optimization, and sampling","abstract":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-08-01","abstract_html":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-08-01","abstract_has_math":false,"creators":["Tzen, Belinda"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Raginsky, Maxim","Jiang, Nan","Rigollet, Philippe","Srikant, Rayadurgam"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-08","date_published":"2022-08","updated_at":"2026-07-22T22:24:55Z","subjects":["machine learning","theory"],"languages":["en","eng"],"rights":["Copyright 2022 Belinda Tzen"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/115878","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Raginsky, Maxim","Jiang, Nan","Rigollet, Philippe","Srikant, Rayadurgam"]},{"key":"dc:creator","label":"Author","values":["Tzen, Belinda"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-08","2022-07-01"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["machine learning","theory"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2022 Belinda Tzen"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/115878"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-08-01","The student, Belinda Tzen, accepted the attached license on 2022-06-28 at 01:16.","The student, Belinda Tzen, submitted this Dissertation for approval on 2022-06-28 at 01:42.","This Dissertation was approved for publication on 2022-07-01 at 07:34.","DSpace SAF Submission Ingestion Package generated from Vireo submission #18113 on 2022-11-16 at 10:17:25","Many problems in machine learning and statistics today exhibit tremendous size in various parameters, to the point that they can often be conceived of as infinite. Doing so enables the use of mathematical tools for continuous systems in their analysis, which entails a careful consideration of the relationship between the discrete system in question and its continuous approximation. The work presented in this thesis uses this approach to study problems in optimization, sampling, and learning, by modeling the phenomena of interest with diffusion processes. In the first part, we bridge a short- and long-timescale understanding of optimization in non-convex settings using a noisy gradient-based method, the Langevin algorithm, a discretization of the eponymous diffusion process. In the second part, we use a control-theoretic framework to examine generative models specified by infinite noisy function compositions, corresponding to nonlinear diffusion processes, and demonstrate their expressiveness as well as methods for performing sampling and inference on them. Subsequently, we study the specific case where these models constitute the infinite-depth limit of feedforward neural networks, and discuss computational aspects of performing variational inference with them. Finally, we investigate probability distributions that can be viewed as the mean-field limit of the weights in very wide neural networks, and contrast control-theoretically optimal and Langevin dynamics vis-\\`a-vis an entropically regularized risk minimization objective. We show why there is an exponential gap in the general case, and close by illustrating a setting where naive gradient-based dynamics in fact coincide exactly with those specified by optimal control: mirror Langevin dynamics, corresponding to the continuous limit of noisy mirror descent."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Applications of diffusion processes: machine learning, optimization, and sampling"]}]}],"canonical_facts":{"dc:contributor":["Raginsky, Maxim","Jiang, Nan","Rigollet, Philippe","Srikant, Rayadurgam"],"dc:creator":["Tzen, Belinda"],"dc:date":["2022-08","2022-07-01"],"dc:description":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-08-01","The student, Belinda Tzen, accepted the attached license on 2022-06-28 at 01:16.","The student, Belinda Tzen, submitted this Dissertation for approval on 2022-06-28 at 01:42.","This Dissertation was approved for publication on 2022-07-01 at 07:34.","DSpace SAF Submission Ingestion Package generated from Vireo submission #18113 on 2022-11-16 at 10:17:25","Many problems in machine learning and statistics today exhibit tremendous size in various parameters, to the point that they can often be conceived of as infinite. Doing so enables the use of mathematical tools for continuous systems in their analysis, which entails a careful consideration of the relationship between the discrete system in question and its continuous approximation. The work presented in this thesis uses this approach to study problems in optimization, sampling, and learning, by modeling the phenomena of interest with diffusion processes. In the first part, we bridge a short- and long-timescale understanding of optimization in non-convex settings using a noisy gradient-based method, the Langevin algorithm, a discretization of the eponymous diffusion process. In the second part, we use a control-theoretic framework to examine generative models specified by infinite noisy function compositions, corresponding to nonlinear diffusion processes, and demonstrate their expressiveness as well as methods for performing sampling and inference on them. Subsequently, we study the specific case where these models constitute the infinite-depth limit of feedforward neural networks, and discuss computational aspects of performing variational inference with them. Finally, we investigate probability distributions that can be viewed as the mean-field limit of the weights in very wide neural networks, and contrast control-theoretically optimal and Langevin dynamics vis-\\`a-vis an entropically regularized risk minimization objective. We show why there is an exponential gap in the general case, and close by illustrating a setting where naive gradient-based dynamics in fact coincide exactly with those specified by optimal control: mirror Langevin dynamics, corresponding to the continuous limit of noisy mirror descent."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/115878"],"dc:language":["en","eng"],"dc:rights":["Copyright 2022 Belinda Tzen"],"dc:subject":["machine learning","theory"],"dc:title":["Applications of diffusion processes: machine learning, optimization, and sampling"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:55Z"}