{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/101231"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/101231","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Towards understanding and simplifying human-in-the-loop machine learning","abstract":"\"Machine learning application developers and data scientists spend inordinate amount of time iterating on machine learning (ML) workflows, by modifying the data pre-processing, model training, and post-processing steps, via trial-and-error to achieve the desired model performance. As a result, developers are \"\"in-the-loop\"\" of the development cycle. Under this \"\"human-in-the-loop\"\" setting, the ultimate goal of a ML system becomes shortening the time to obtain deployable models from scratch. However, some of the existing ML systems ignore this iterative aspect, and only optimize the one-shot execution of the workflow, while some of them don't provide enough support for system users to make iterative changes. Here, we first conduct a mini-survey of the applied machine learning literature to quantitatively study the user behavior in iterative ML application development. Then, we propose Helix, a declarative machine learning system implemented in Scala. Helix mainly focuses on the optimization of the execution across iterations by reusing or recomputing intermediate results as appropriate. Finally, we describe our collaboration system on top of Helix, that includes a workflow management module and a visualization tool, to make the machine learning system easier to use. In our evaluations, Helix achieved a 60% magnitude reduction in cumulative running time compared to state-of-the-art machine learning tools.\"","abstract_html":""Machine learning application developers and data scientists spend inordinate amount of time iterating on machine learning (ML) workflows, by modifying the data pre-processing, model training, and post-processing steps, via trial-and-error to achieve the desired model performance. As a result, developers are ""in-the-loop"" of the development cycle. Under this ""human-in-the-loop"" setting, the ultimate goal of a ML system becomes shortening the time to obtain deployable models from scratch. However, some of the existing ML systems ignore this iterative aspect, and only optimize the one-shot execution of the workflow, while some of them don't provide enough support for system users to make iterative changes. Here, we first conduct a mini-survey of the applied machine learning literature to quantitatively study the user behavior in iterative ML application development. Then, we propose Helix, a declarative machine learning system implemented in Scala. Helix mainly focuses on the optimization of the execution across iterations by reusing or recomputing intermediate results as appropriate. Finally, we describe our collaboration system on top of Helix, that includes a workflow management module and a visualization tool, to make the machine learning system easier to use. In our evaluations, Helix achieved a 60% magnitude reduction in cumulative running time compared to state-of-the-art machine learning tools."","abstract_has_math":false,"creators":["Ma, Litian"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Parameswaran, Aditya"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2018,"date_issued":"2018-09-04T20:41:58Z","date_published":"2018-09-04T20:41:58Z","updated_at":"2026-07-22T22:24:38Z","subjects":["human in the loop computing","machine learning"],"languages":["en"],"rights":["Copyright 2018 Litian Ma"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/101231","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Parameswaran, Aditya"]},{"key":"dc:creator","label":"Author","values":["Ma, Litian"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2018-09-04T20:41:58Z","2020-09-05T09:15:13Z","2018-04-26","2018-05"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["human in the loop computing","machine learning"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2018 Litian Ma"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/101231"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["\"Machine learning application developers and data scientists spend inordinate amount of time iterating on machine learning (ML) workflows, by modifying the data pre-processing, model training, and post-processing steps, via trial-and-error to achieve the desired model performance. As a result, developers are \"\"in-the-loop\"\" of the development cycle. Under this \"\"human-in-the-loop\"\" setting, the ultimate goal of a ML system becomes shortening the time to obtain deployable models from scratch. However, some of the existing ML systems ignore this iterative aspect, and only optimize the one-shot execution of the workflow, while some of them don't provide enough support for system users to make iterative changes. Here, we first conduct a mini-survey of the applied machine learning literature to quantitatively study the user behavior in iterative ML application development. Then, we propose Helix, a declarative machine learning system implemented in Scala. Helix mainly focuses on the optimization of the execution across iterations by reusing or recomputing intermediate results as appropriate. Finally, we describe our collaboration system on top of Helix, that includes a workflow management module and a visualization tool, to make the machine learning system easier to use. In our evaluations, Helix achieved a 60% magnitude reduction in cumulative running time compared to state-of-the-art machine learning tools.\"","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2020-05-01","The student, Litian Ma, accepted the attached license on 2018-04-25 at 20:27.","The student, Litian Ma, submitted this Thesis for approval on 2018-04-25 at 20:28.","This Thesis was approved for publication on 2018-04-26 at 09:01.","DSpace SAF Submission Ingestion Package generated from Vireo submission #12508 on 2018-08-31 at 17:21:34","Made available in DSpace on 2018-09-04T20:41:58Z (GMT). No. of bitstreams: 3 MA-THESIS-2018.pdf: 1940500 bytes, checksum: 56decd2f5bb97ad45eda64078970cf6f (MD5) UIUCGraduateThesis-master.zip: 7812584 bytes, checksum: 64980208eeb010e16294ee2a53d23a2b (MD5) LICENSE.txt: 4206 bytes, checksum: b45654ad0207ba8fe8e23231ac95d97e (MD5) Previous issue date: 2018-04-26","Embargo set by: Seth Robbins for item 107316 Lift date: 2020-09-04T20:42:08Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only Restriction Lifted for Item 107316 on 2020-09-05T09:15:13Z."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Towards understanding and simplifying human-in-the-loop machine learning"]}]}],"canonical_facts":{"dc:contributor":["Parameswaran, Aditya"],"dc:creator":["Ma, Litian"],"dc:date":["2018-09-04T20:41:58Z","2020-09-05T09:15:13Z","2018-04-26","2018-05"],"dc:description":["\"Machine learning application developers and data scientists spend inordinate amount of time iterating on machine learning (ML) workflows, by modifying the data pre-processing, model training, and post-processing steps, via trial-and-error to achieve the desired model performance. As a result, developers are \"\"in-the-loop\"\" of the development cycle. Under this \"\"human-in-the-loop\"\" setting, the ultimate goal of a ML system becomes shortening the time to obtain deployable models from scratch. However, some of the existing ML systems ignore this iterative aspect, and only optimize the one-shot execution of the workflow, while some of them don't provide enough support for system users to make iterative changes. Here, we first conduct a mini-survey of the applied machine learning literature to quantitatively study the user behavior in iterative ML application development. Then, we propose Helix, a declarative machine learning system implemented in Scala. Helix mainly focuses on the optimization of the execution across iterations by reusing or recomputing intermediate results as appropriate. Finally, we describe our collaboration system on top of Helix, that includes a workflow management module and a visualization tool, to make the machine learning system easier to use. In our evaluations, Helix achieved a 60% magnitude reduction in cumulative running time compared to state-of-the-art machine learning tools.\"","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2020-05-01","The student, Litian Ma, accepted the attached license on 2018-04-25 at 20:27.","The student, Litian Ma, submitted this Thesis for approval on 2018-04-25 at 20:28.","This Thesis was approved for publication on 2018-04-26 at 09:01.","DSpace SAF Submission Ingestion Package generated from Vireo submission #12508 on 2018-08-31 at 17:21:34","Made available in DSpace on 2018-09-04T20:41:58Z (GMT). No. of bitstreams: 3 MA-THESIS-2018.pdf: 1940500 bytes, checksum: 56decd2f5bb97ad45eda64078970cf6f (MD5) UIUCGraduateThesis-master.zip: 7812584 bytes, checksum: 64980208eeb010e16294ee2a53d23a2b (MD5) LICENSE.txt: 4206 bytes, checksum: b45654ad0207ba8fe8e23231ac95d97e (MD5) Previous issue date: 2018-04-26","Embargo set by: Seth Robbins for item 107316 Lift date: 2020-09-04T20:42:08Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only Restriction Lifted for Item 107316 on 2020-09-05T09:15:13Z."],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/101231"],"dc:language":["en"],"dc:rights":["Copyright 2018 Litian Ma"],"dc:subject":["human in the loop computing","machine learning"],"dc:title":["Towards understanding and simplifying human-in-the-loop machine learning"],"dc:type":["text"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:38Z"}