New capabilities for large-scale exploratory data analysis

The ever-rising diversity of data generated, manipulated, and analyzed every day engenders a variety of data formats, ranging from one fixed dataset to multiple versions of a dataset stored across multiple data sources. This variety of formats has led to substantial challenges in data exploration. Existing systems do not effectively support querying capabilities across these formats: (i) Browsing: When exploring a single dataset, data scientists often need to examine a collection of records that satisfy arbitrary predicates. However, current exploratory data analysis tools mainly focus on visual summarization over browsing. (ii) Versioning: With the proliferation of dataset versions generated during different stages of exploration, exploratory data analysis is no longer just about exploring one static dataset. Instead, data scientists need to keep track of massive numbers of versions, as well as search for versions with specific criteria. (iii) Integrating: Nowadays, datasets are collected and stored at multiple sources (e.g., as part of the IoT). When exploring data, data scientists often need to query and join data across databases at disparate locations. In this dissertation, we propose systems that enable query capabilities to efficiently and effectively fulfill these new demands in data exploration. (i) For browsing, we develop NEEDLETAIL, a data exploration engine that employs a light-weight indexing structure along with efficient algorithms to retrieve any-k valid records for arbitrary queries as quickly as possible. (ii) For versioning, we implement and open-source ORPHEUSDB, a dataset version control system that can efficiently track and query across dataset versions. Since versioning queries in ORPHEUSDB take advantage of array operators in relational database systems, we also conduct an extensive experimental study on understanding array implementations in modern database systems. (iii) For integrating, we leverage machine learning techniques to optimize federated query processing and eventually improve the interactivity of data exploration across disparate databases.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2020

Author dc:creator

• Xu, Liqi

Contributors dc:contributor

• Parameswaran, Aditya
• Zhai, ChengXiang
• Tao, Xie
• Cole, Richard L.

• Exploratory Data Analysis
• Data Management

Statement dc:rights

• Copyright 2020 Liqi Xu

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/107971

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/107971