{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/127492"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/127492","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Reliable and efficient machine learning under distribution shifts","abstract":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-12-01","abstract_html":"Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-12-01","abstract_has_math":false,"creators":["Wang, Haoxiang"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Zhao, Han","Li, Bo","Schwing, Alexander Gerhard","Raginsky, Maxim"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-12-06","date_published":"2024-12-06","updated_at":"2026-07-22T22:25:04Z","subjects":["Machine Learning","Deep Learning","Distribution Shift"],"languages":["en","eng"],"rights":["Copyright 2024 Haoxiang Wang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/127492","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Zhao, Han","Li, Bo","Schwing, Alexander Gerhard","Raginsky, Maxim"]},{"key":"dc:creator","label":"Author","values":["Wang, Haoxiang"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2024-12-06","2024-12"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Machine Learning","Deep Learning","Distribution Shift"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2024 Haoxiang Wang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/127492"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-12-01","The student, Haoxiang Wang, accepted the attached license on 2024-12-04 at 01:05.","The student, Haoxiang Wang, submitted this Dissertation for approval on 2024-12-04 at 01:36.","This Dissertation was approved for publication on 2024-12-06 at 08:07.","DSpace SAF Submission Ingestion Package generated from Vireo submission #21490 on 2025-03-28 at 14:55:46","Over the past decade, machine learning has emerged as a transformative technology, driving advancements across various domains including computer vision, natural language processing, biology, autonomous driving, and recommendation systems. These innovations have been fueled by the development of increasingly sophisticated algorithms, the availability of large-scale datasets, and the growth of computational power. However, a crucial challenge persists in applying machine learning models to real-world scenarios: their performance tends to degrade significantly when confronted with distribution shifts. This dissertation primarily focuses on two types of distribution shifts: task distribution shifts and data distribution shifts. In addressing task distribution shifts, we first develop a theoretical framework to analyze gradient-based meta-learning algorithms from an optimization perspective, leveraging the Neural Tangent Kernel (NTK) as a theoretical tool. We then bridge multi-task learning (MTL) and meta-learning within this NTK-based framework, providing a unified understanding of these learning paradigms. Finally, we tackle a real-world challenge of aligning large language models with diverse user preferences by developing a novel MTL algorithm called Directional Preference Alignment. Regarding data distribution shifts, we first investigate the paradigm of gradual domain adaptation (GDA) through theoretical analysis. This leads to a deeper understanding of GDA and inspires a novel algorithm: Generative Gradual Domain Adaptation with Optimal Transport. Further, we consider the domain generalization setup and develop a new class of provable algorithms, Invariant-feature Subspace Recovery, which can be used to mitigate spurious correlations learned by machine learning models. Finally, we introduce a previously unexplored challenge of data distribution shifts, which we term compositional generalization in the context of multi-domain multi-class classification. To address this challenge, we design a theory-principled fine-tuning algorithm, Compositional Feature Alignment, aimed at enhancing the compositional generalization ability of pre-trained models. Through these contributions, this dissertation advances the field of machine learning by improving model reliability and efficiency under task and data distribution shifts, and bridges the gap between theoretical frameworks and practical applications in various real-world application scenarios."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Reliable and efficient machine learning under distribution shifts"]}]}],"canonical_facts":{"dc:contributor":["Zhao, Han","Li, Bo","Schwing, Alexander Gerhard","Raginsky, Maxim"],"dc:creator":["Wang, Haoxiang"],"dc:date":["2024-12-06","2024-12"],"dc:description":["Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2026-12-01","The student, Haoxiang Wang, accepted the attached license on 2024-12-04 at 01:05.","The student, Haoxiang Wang, submitted this Dissertation for approval on 2024-12-04 at 01:36.","This Dissertation was approved for publication on 2024-12-06 at 08:07.","DSpace SAF Submission Ingestion Package generated from Vireo submission #21490 on 2025-03-28 at 14:55:46","Over the past decade, machine learning has emerged as a transformative technology, driving advancements across various domains including computer vision, natural language processing, biology, autonomous driving, and recommendation systems. These innovations have been fueled by the development of increasingly sophisticated algorithms, the availability of large-scale datasets, and the growth of computational power. However, a crucial challenge persists in applying machine learning models to real-world scenarios: their performance tends to degrade significantly when confronted with distribution shifts. This dissertation primarily focuses on two types of distribution shifts: task distribution shifts and data distribution shifts. In addressing task distribution shifts, we first develop a theoretical framework to analyze gradient-based meta-learning algorithms from an optimization perspective, leveraging the Neural Tangent Kernel (NTK) as a theoretical tool. We then bridge multi-task learning (MTL) and meta-learning within this NTK-based framework, providing a unified understanding of these learning paradigms. Finally, we tackle a real-world challenge of aligning large language models with diverse user preferences by developing a novel MTL algorithm called Directional Preference Alignment. Regarding data distribution shifts, we first investigate the paradigm of gradual domain adaptation (GDA) through theoretical analysis. This leads to a deeper understanding of GDA and inspires a novel algorithm: Generative Gradual Domain Adaptation with Optimal Transport. Further, we consider the domain generalization setup and develop a new class of provable algorithms, Invariant-feature Subspace Recovery, which can be used to mitigate spurious correlations learned by machine learning models. Finally, we introduce a previously unexplored challenge of data distribution shifts, which we term compositional generalization in the context of multi-domain multi-class classification. To address this challenge, we design a theory-principled fine-tuning algorithm, Compositional Feature Alignment, aimed at enhancing the compositional generalization ability of pre-trained models. Through these contributions, this dissertation advances the field of machine learning by improving model reliability and efficiency under task and data distribution shifts, and bridges the gap between theoretical frameworks and practical applications in various real-world application scenarios."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/127492"],"dc:language":["en","eng"],"dc:rights":["Copyright 2024 Haoxiang Wang"],"dc:subject":["Machine Learning","Deep Learning","Distribution Shift"],"dc:title":["Reliable and efficient machine learning under distribution shifts"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:04Z"}