Visualizations for model tracking and predictions in machine learning

Building machine learning models is often an exploratory and iterative process. A data scientist frequently builds and trains hundreds of models with different parameters and feature sets in order to find one that meets the desired criteria. However, it can be difficult to keep track of all the parameters and metadata that are associated with the models. ModelDB, an end-to-end system for managing machine learning models, is a tool that solves this problem of model management. In this thesis, we present a graphical user interface for ModelDB, along with an extension for visualizing model predictions. The core user interface for model management augments the ModelDB system, which previously consisted only of native client libraries and a backend. The interface provides new ways of exploring, visualizing, and analyzing model data through a web application. The prediction visualizations extend the core user interface by providing a novel prediction matrix that displays classifier outputs in order to convey model performance at the example level. We present the design and implementation of both the core user interface and the prediction visualizations, discussing at each step the motivations behind key features. We evaluate the prediction visualizations through a pilot user study, which produces preliminary feedback on the practicality and utility of the interface. The overall goal of this research is to provide a powerful, user-friendly interface that leverages the data stored in ModelDB to generate effective visualizations for analyzing and improving models.

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2017

Author dc:creator

• Lee, Wei-En

Advisor dc:contributor.advisor

• Samuel Madden.

• Electrical Engineering and Computer Science.

Statement dc:rights

• MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/113133

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/113133