A flexible framework for composing end to end machine learning pipelines

In this thesis, we aim to simplify the building of end-to-end machine learning pipelines while preserving the performance of such pipelines on real data. As a solution to this, we propose the MLBlocks framework, a system that allows an end user to obtain a pipeline with only data and a list of data science blocks. Once a pipeline is specified, a user can tune its hyper-parameters, as well as fit and predictions, with minimal code. When building MLBlocks, we first develop a data science block library that seamlessly integrates third party blocks without integration code, providing a foundation for users to start building data science pipelines. We then provide the MLPipeline framework that allows users to simply tie together these blocks and perform the aforementioned tuning, fitting, and predicting operations with the resulting pipelines. Finally, we test the framework's usability as well as its ability to preserve performance on real data by running several pipelines on various data modalities and by integrating MLBlocks into a larger scale project. Since we are able to replicate the pipelines already in use, we are able to obtain identical results while dramatically simplifying application logic. We conclude that MLBlocks succeeds in providing a simple but effective solution to making the construction of high-performing end-to-end pipelines both simpler and more accessible.

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2018

Author dc:creator

• Xue, William, M. Eng. Massachusetts Institute of Technology

Advisor dc:contributor.advisor

• Kalyan Veeramachaneni.

• Electrical Engineering and Computer Science.

Statement dc:rights

• MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/119550

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/119550