Learning distributions with Particle Mirror Descent

As an effective and provable primal method to estimate posterior distribution, Particle Mirror Descent is appealing for its simplicity and flexibility. In this thesis we explore the applications of Particle Mirror Descent in both supervised and unsupervised learning. In the general classification problem with a parametric discriminative function, we seek a posterior distribution over parameters that maximizes classification accuracy. Existing algorithms usually solve the dual problem and the number of variables to optimize depends on the number of examples in the dataset. Therefore such methods suffer from the poor scalability in large datasets. We propose Constrained Particle Mirror Descent to effectively estimate posterior distribution in primal space even with expectation constraint. By marrying Bayesian probabilistic inference and deep neural networks, deep generative networks have shown remarkable success in various kinds of generative tasks. However, such models usually make an assumption that posterior distribution can be simply characterized as a Gaussian distribution, which is not always true since real data like images and audios yield complex structures in latent space. Motivated by the recent wide application of multi-modal posterior, we introduce a variant of Variational Auto-encoder model that uses a mixture of customized kernels as posterior distribution in latent space. Our deep generative model produces visually plausible images as well as good clustering performance using latent representations.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2017

Author dc:creator

• Zhang, Ailing

Contributors dc:contributor

• Koyejo, Oluwasanmi
• He, Niao

• Constrained Particle Mirror Descent
• Deep generative networks

Statement dc:rights

• Copyright 2017 Ailing Zhang

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/97411

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/97411