Geometric and functional representations of stochastic neural dynamical systems: from realization theory to controlled approximation

There has been a great deal of interest in understanding continuous-time processes in deep learning, in particular methods related to (stochastic) control for improving diffusion models. In this work, we explore various facets of function approximation and realization problems through the lens of dynamical systems theory, neural stochastic differential equations (neural SDEs), and differential geometry. A neural SDE is an Itô diffusion process whose drift and diffusion matrices are elements of some parametric families. We cover topics from estimating the transition density of both uniformly elliptic and possibly degenerate diffusion processes by leveraging tools from sub-Riemannian geometry and stochastic control. There are many nuanced insights we can get into the behavior of deep neural networks and diffusion models by studying properties of associated problems in optimal control theory and drawing on other tools from the rich mathematical physics literature. The geometric insights explain the underlying noise structure and controllability properties of a stochastic dynamical system while also explaining the expressive power of the stochastic system.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Veeravalli, Tanya

Contributors dc:contributor

• Raginsky, Maxim
• Srikant, Rayadurgam
• Belabbas, Mohamed Ali
• Zhao, Zhizhen

• Optimal Control
• Neural Dynamical Systems
• Stochastic Realization Theory
• Nonlinear Controllability
• Function Approximation

Statement dc:rights

• Copyright 2025 Tanya Veeravalli

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/129887

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F129887