Bayesian tuning and bandits : an extensible, open source library for AutoML

The goal of this thesis is to build an extensible and open source library that handles the problems of tuning the hyperparameters of a machine learning pipeline, selecting between multiple pipelines, and recommending a pipeline. We devise a library that users can integrate into their existing datascience workflows and experts can contribute to by writing methods to solve these search problems. Extending upon the existing library, our goals are twofold: one that the library naturally fits within a user's existing workflow, so that integration does not require a lot of overhead, and two that the three search problems are broken down into small and modular pieces to allow contributors to have maximal flexibility. We establish the abstractions for each of the solutions to these search problems, showcasing how both a user would use the library and a contributor could override the API. We discuss the creation of a recommender system, that proposes machine learning pipelines for a new dataset, trained on an existing matrix of known scores of pipelines on datasets. We show how using such a system can lead to performance gains. We discuss how we can evaluate the quality of different solutions to these types of search problems, and how we can measurably compare them to each other.

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2018

Author dc:creator

• Gustafson, Laura (Laura N.)

Advisor dc:contributor.advisor

• Kalyan Veeramachaneni.

• Electrical Engineering and Computer Science.

Statement dc:rights

• MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/119764

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/119764