Parallel and distributed MCMC inference using Julia

Machine learning algorithms are often computationally intensive and operate on large datasets. Being able to eciently learn models on large datasets holds the future of machine learning. As the speed of serial computation stalls, it is necessary to utilize the power of parallel computing in order to better scale with the growing complexity of algorithms and the growing size of datasets. In this thesis, we explore the use of Julia, a fairly new high level programming language that lends itself to easy parallelization over multiple CPU cores as well as multiple machines, on Markov chain Monte Carlo (MCMC) inference algorithms. First, we take existing algorithms and implement them in Julia. We focus on MCMC inference using Continuous Piecewise-Affine Based (CPAB) transformations and a parallel MCMC sampler for Dirichlet Process Mixture Models (DPMM). Instead of parallelizing over multiple cores on a single machine, our Julia implementations extend existing implementations by parallelizing over multiple machines. We compare our implementation with these existing implementations written in more traditional programming languages. Next, we develop a model Projections Dirichlet Process Gaussian Mixture Model (PDP-GMM) which relaxes the assumption that the draws from a Dirichlet Process Gaussian Mixture Model (DP-GMM) are directly observed. We extend our DPMM Julia implementation and present a few applications of this model.

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2016

Author dc:creator

• Yu, Angel

Advisor dc:contributor.advisor

• John W. Fisher III and Oren Freifeld.

• Electrical Engineering and Computer Science.

Statement dc:rights

• MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/113440

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/113440