{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/105669"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/105669","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Effective data versioning for collaborative data analytics","abstract":"With the massive proliferation of datasets in a variety of sectors, data science teams in these sectors spend vast amounts of time collaboratively constructing, curating, and analyzing these datasets. Versions of datasets are routinely generated during this data science process, via various data processing operations like data transformation and cleaning, feature engineering and normalization, among others. However, no existing systems enable us to effectively store, track, and query these versioned datasets, leading to massive redundancy in versioned data storage and making true collaboration and sharing impossible. In this thesis, we develop solutions for versioned data management for collaborative data analytics. In the first part of this thesis, we extend a relational database to support versioning of structured data. Specifically, we build a system, OrpheusDB, on top of a relational database with a carefully designed data representation and an intelligent partitioning algorithm for fast version control operations. OrpheusDB inherits much of the same benefits of relational databases, while also compactly storing, keeping track of, and recreating versions on demand. However, OrpheusDB implicitly makes a few assumptions, namely that: (a) the SQL assumption: a SQL-like language is the best fit for querying data and versioning information; (b) the structural assumption: the data is in a relational format with a regular structure; (c) the from-scratch assumption: users adopt OrpheusDB from the very beginning of their project and register each data version along with full metadata in the system. In the second part of this thesis, we remove each of these assumptions, one at a time. First, we remove the SQL assumption and propose a generalized query language for querying data along with versioning and provenance information. Second, we remove the structural assumption and develop solutions for compact storage and fast retrieval of arbitrary data representations. Finally, we remove the “from-scratch” assumption, by developing techniques to infer lineage relationships among versions residing in an existing data repository.","abstract_html":"With the massive proliferation of datasets in a variety of sectors, data science teams in these sectors spend vast amounts of time collaboratively constructing, curating, and analyzing these datasets. Versions of datasets are routinely generated during this data science process, via various data processing operations like data transformation and cleaning, feature engineering and normalization, among others. However, no existing systems enable us to effectively store, track, and query these versioned datasets, leading to massive redundancy in versioned data storage and making true collaboration and sharing impossible. In this thesis, we develop solutions for versioned data management for collaborative data analytics. In the first part of this thesis, we extend a relational database to support versioning of structured data. Specifically, we build a system, OrpheusDB, on top of a relational database with a carefully designed data representation and an intelligent partitioning algorithm for fast version control operations. OrpheusDB inherits much of the same benefits of relational databases, while also compactly storing, keeping track of, and recreating versions on demand. However, OrpheusDB implicitly makes a few assumptions, namely that: (a) the SQL assumption: a SQL-like language is the best fit for querying data and versioning information; (b) the structural assumption: the data is in a relational format with a regular structure; (c) the from-scratch assumption: users adopt OrpheusDB from the very beginning of their project and register each data version along with full metadata in the system. In the second part of this thesis, we remove each of these assumptions, one at a time. First, we remove the SQL assumption and propose a generalized query language for querying data along with versioning and provenance information. Second, we remove the structural assumption and develop solutions for compact storage and fast retrieval of arbitrary data representations. Finally, we remove the “from-scratch” assumption, by developing techniques to infer lineage relationships among versions residing in an existing data repository.","abstract_has_math":false,"creators":["Huang, Silu"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Parameswaran, Aditya","Han, Jiawei","Sinha, Saurabh","Elmore, Aaron"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2019,"date_issued":"2019-11-26T20:34:59Z","date_published":"2019-11-26T20:34:59Z","updated_at":"2026-07-22T22:24:44Z","subjects":["Data Versioning, Collaborative Data Analytics, Partitioning, Data Representation"],"languages":["en"],"rights":["Copyright 2019 Silu Huang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/105669","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Parameswaran, Aditya","Han, Jiawei","Sinha, Saurabh","Elmore, Aaron"]},{"key":"dc:creator","label":"Author","values":["Huang, Silu"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2019-11-26T20:34:59Z","2019-07-11","2019-08"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Data Versioning, Collaborative Data Analytics, Partitioning, Data Representation"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2019 Silu Huang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/105669"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["With the massive proliferation of datasets in a variety of sectors, data science teams in these sectors spend vast amounts of time collaboratively constructing, curating, and analyzing these datasets. Versions of datasets are routinely generated during this data science process, via various data processing operations like data transformation and cleaning, feature engineering and normalization, among others. However, no existing systems enable us to effectively store, track, and query these versioned datasets, leading to massive redundancy in versioned data storage and making true collaboration and sharing impossible. In this thesis, we develop solutions for versioned data management for collaborative data analytics. In the first part of this thesis, we extend a relational database to support versioning of structured data. Specifically, we build a system, OrpheusDB, on top of a relational database with a carefully designed data representation and an intelligent partitioning algorithm for fast version control operations. OrpheusDB inherits much of the same benefits of relational databases, while also compactly storing, keeping track of, and recreating versions on demand. However, OrpheusDB implicitly makes a few assumptions, namely that: (a) the SQL assumption: a SQL-like language is the best fit for querying data and versioning information; (b) the structural assumption: the data is in a relational format with a regular structure; (c) the from-scratch assumption: users adopt OrpheusDB from the very beginning of their project and register each data version along with full metadata in the system. In the second part of this thesis, we remove each of these assumptions, one at a time. First, we remove the SQL assumption and propose a generalized query language for querying data along with versioning and provenance information. Second, we remove the structural assumption and develop solutions for compact storage and fast retrieval of arbitrary data representations. Finally, we remove the “from-scratch” assumption, by developing techniques to infer lineage relationships among versions residing in an existing data repository.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2019-11-26 without embargo terms","The student, Silu Huang, accepted the attached license on 2019-07-10 at 13:20.","The student, Silu Huang, submitted this Dissertation for approval on 2019-07-10 at 13:29.","This Dissertation was approved for publication on 2019-07-11 at 16:59.","DSpace SAF Submission Ingestion Package generated from Vireo submission #14246 on 2019-11-26 at 12:52:15","Made available in DSpace on 2019-11-26T20:34:59Z (GMT). No. of bitstreams: 4 HUANG-DISSERTATION-2019.pdf: 3907603 bytes, checksum: f1e2544ba3fa9dd9973319f299db037a (MD5) siluhuang-thesis.zip: 13170435 bytes, checksum: 6fa405acbc46f2893b9946c1876a309e (MD5) LICENSE.txt: 4207 bytes, checksum: cb7554818650eddcbf83a3f9e19d213d (MD5) PROQUEST_LICENSE.txt: 4553 bytes, checksum: fcb7d60288ef3a01d0c36081d26cdc76 (MD5) Previous issue date: 2019-07-11"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Effective data versioning for collaborative data analytics"]}]}],"canonical_facts":{"dc:contributor":["Parameswaran, Aditya","Han, Jiawei","Sinha, Saurabh","Elmore, Aaron"],"dc:creator":["Huang, Silu"],"dc:date":["2019-11-26T20:34:59Z","2019-07-11","2019-08"],"dc:description":["With the massive proliferation of datasets in a variety of sectors, data science teams in these sectors spend vast amounts of time collaboratively constructing, curating, and analyzing these datasets. Versions of datasets are routinely generated during this data science process, via various data processing operations like data transformation and cleaning, feature engineering and normalization, among others. However, no existing systems enable us to effectively store, track, and query these versioned datasets, leading to massive redundancy in versioned data storage and making true collaboration and sharing impossible. In this thesis, we develop solutions for versioned data management for collaborative data analytics. In the first part of this thesis, we extend a relational database to support versioning of structured data. Specifically, we build a system, OrpheusDB, on top of a relational database with a carefully designed data representation and an intelligent partitioning algorithm for fast version control operations. OrpheusDB inherits much of the same benefits of relational databases, while also compactly storing, keeping track of, and recreating versions on demand. However, OrpheusDB implicitly makes a few assumptions, namely that: (a) the SQL assumption: a SQL-like language is the best fit for querying data and versioning information; (b) the structural assumption: the data is in a relational format with a regular structure; (c) the from-scratch assumption: users adopt OrpheusDB from the very beginning of their project and register each data version along with full metadata in the system. In the second part of this thesis, we remove each of these assumptions, one at a time. First, we remove the SQL assumption and propose a generalized query language for querying data along with versioning and provenance information. Second, we remove the structural assumption and develop solutions for compact storage and fast retrieval of arbitrary data representations. Finally, we remove the “from-scratch” assumption, by developing techniques to infer lineage relationships among versions residing in an existing data repository.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2019-11-26 without embargo terms","The student, Silu Huang, accepted the attached license on 2019-07-10 at 13:20.","The student, Silu Huang, submitted this Dissertation for approval on 2019-07-10 at 13:29.","This Dissertation was approved for publication on 2019-07-11 at 16:59.","DSpace SAF Submission Ingestion Package generated from Vireo submission #14246 on 2019-11-26 at 12:52:15","Made available in DSpace on 2019-11-26T20:34:59Z (GMT). No. of bitstreams: 4 HUANG-DISSERTATION-2019.pdf: 3907603 bytes, checksum: f1e2544ba3fa9dd9973319f299db037a (MD5) siluhuang-thesis.zip: 13170435 bytes, checksum: 6fa405acbc46f2893b9946c1876a309e (MD5) LICENSE.txt: 4207 bytes, checksum: cb7554818650eddcbf83a3f9e19d213d (MD5) PROQUEST_LICENSE.txt: 4553 bytes, checksum: fcb7d60288ef3a01d0c36081d26cdc76 (MD5) Previous issue date: 2019-07-11"],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/105669"],"dc:language":["en"],"dc:rights":["Copyright 2019 Silu Huang"],"dc:subject":["Data Versioning, Collaborative Data Analytics, Partitioning, Data Representation"],"dc:title":["Effective data versioning for collaborative data analytics"],"dc:type":["text"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:44Z"}