Towards an automatic predictive question formulation

In this thesis, we designed a formal language, called Trane, for describing prediction problems over relational datasets, implemented a system that allows humans to specify problems in that language, and allows them to build models that solve them using real data. We show that this language is able to describe all 54 prediction problems on the Kaggle data science competition website[14] and so is comprehensive. The implemented system consists of a web application connected to a server-side interpreter, which translates input from the web application into a series of transformation and aggregation operations to apply to a dataset in order to generate labels that can be used to train a supervised machine learning classifier. Using a smaller subset of this language, we developed software that enumerated 1077 prediction problems automatically for the Walmart Store Sales Forecasting dataset found on Kaggle[16], and built models that attempted to solve them, for which we produced 235 AUC scores. The web application also allowed us to collect 157 ratings from humans on the meaningfulness of randomly-generated prediction problems. We used these ratings along with an enumeration of 6105 prediction problems and 7 datasets to train a collaborative-filtering based recommendation system to propose meaningful prediction problems on new, unseen datasets.

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2016

Author dc:creator

• Schreck, Benjamin J

Advisor dc:contributor.advisor

• Kalyan Veeramachaneni.

• Electrical Engineering and Computer Science.

Statement dc:rights

• M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/105963

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/105963