Optimization in the space of probability distributions with applications in statistics

Many problems in statistics can be formulated as minimizing a functional over the space of all probability distributions on a (parameter) space. Examples include approximate Bayesian computation, non-parametric estimation, and single-cell analysis in mathematical biology. This thesis aims to address the following statistical problems by developing computationally tractable algorithms with theoretical foundations: 1. In the first part, we introduce a general computational framework to implement mean-field (MF) variational inference (VI) for Bayesian models using the Wasserstein gradient flow (WGF), a modern mathematical technique for realizing a gradient flow over the space of probability distributions. We prove the statistical guarantee of MFVI and the algorithmic convergence of using WGF under mild conditions. To implement the algorithm, we propose a new constraint-free function approximation method using neural networks. 2. In the second part, we investigate the nonparametric estimation problem of estimating the distribution-state dependent drift parameter of a Mckean--Vlasov equation with a constant diffusion parameter. This equation can be treated as the stochastic differential equation of which the density evolution equation is the gradient flow to minimize a specific energy functional. 3. In the third part, we concern the optimization of a displacement convex functional over multiple distributions. We derive an exponential algorithmic convergence rate under the quadratic growth (QG) condition on the objective functional, and a slower polynomial rate in the absence of the QG condition. By applying our theory to MFVI, the dependence of the algorithmic convergence rate on the number of blocks is better compared with the result in the first project. 4. In the last part, we explore the problem of convex optimization over the space of all probability distributions. We introduce an implicit scheme for discretizing a continuous-time gradient flow relative to the Kullback--Leibler (KL) divergence. We derive an explicit algorithmic convergence rate and apply our algorithm to compute non-parametric likelihood estimators and Bayesian posterior distributions.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Statistics

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2024

Author dc:creator

• Yao, Rentian

Contributors dc:contributor

• Yang, Yun
• Chen, Xiaohui
• Shao, Xiaofeng
• Liu, Jingbo

• Optimal Transport
• Bayesian Statistics
• Interacting Particle Systems
• Optimization

Statement dc:rights

• Copyright 2024 Rentian Yao

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/125655

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F125655