On the Inductive Biases of Conditional Diffusion Models

Diffusion models have achieved remarkable progress in recent years across various domains and applications, but how diffusion models generalize is still not well understood. While prior work predominantly focuses on unconditional diffusion models, in this thesis we focus on understanding generalization for conditional diffusion models, which is especially relevant for modern text- or observation- conditioned applications. In particular, we are interested in the inductive biases of conditional diffusion models which predispose them to certain forms of interpolation in regions outside the support of the training data. We observe that neural networks are capable of learning qualitatively different forms of interpolation, which may be influenced by the architecture and capacity of the network and other aspects of neural network training. We develop a potential framework to model the interpolation behavior of neural networks via nonparametric estimation, which happens to have the property of being schedule consistent, or truly denoising at every time step. We find that, assuming a neural network with sufficient capacity, conditional diffusion models are biased towards smoothing, which can lead to non-schedule consistent behavior away from the training data and has a number of interesting consequences.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2025

Author dc:creator

• Yu, Christina

Advisor dc:contributor.advisor

• Tedrake, Russ

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright retained by author(s)

Licence dc:rights.uri

• https://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/159081

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/159081