{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/120084"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/120084","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Efficient invariant feature subspace recovery for domain generalization","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-09-01 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-09-01 without embargo terms","abstract_has_math":false,"creators":["Balasubramaniam, Gargi"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Zhao, Han"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2023,"date_issued":"2023-05","date_published":"2023-05","updated_at":"2026-07-22T22:24:56Z","subjects":["Domain Generalization","Out-of-domain (OOD) Generalization","Invariant Feature Learning","Spurious Correlations"],"languages":["en","eng"],"rights":["Copyright 2023 Gargi Balasubramaniam"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/120084","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Zhao, Han"]},{"key":"dc:creator","label":"Author","values":["Balasubramaniam, Gargi"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2023-05","2023-04-24"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Domain Generalization","Out-of-domain (OOD) Generalization","Invariant Feature Learning","Spurious Correlations"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2023 Gargi Balasubramaniam"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/120084"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-09-01 without embargo terms","The student, Gargi Balasubramaniam, accepted the attached license on 2023-04-23 at 12:57.","The student, Gargi Balasubramaniam, submitted this Thesis for approval on 2023-04-23 at 13:04.","This Thesis was approved for publication on 2023-04-24 at 11:53.","DSpace SAF Submission Ingestion Package generated from Vireo submission #19121 on 2023-09-01 at 16:55:08","Domain generalization aims to learn a machine learning model over multiple training environments to generalize well to unseen test environments. With the advent of large pretrained models, recent work focuses on efficient domain generalization to maximize performance under minimal data and model fine-tuning requirements. Recently, Wang et al. proposed Invariant Feature Subspace Recovery (ISR-Mean), an efficient domain generalization algorithm which uses means of the class-conditional data distributions to provably identify the domain-invariant feature subspace under a given causal model. However, ISR-Mean is not applicable in the realistic setting of multi-class classification as it only utilizes information from a single class, failing to account for information across multiple classes. Further, it is unclear how this notion of invariant subspace recovery can be applied to regression, when there are no class labels. Motivated by the need to enable efficient robustness via invariant feature subspace recovery in these scenarios, this work extends ISR to two novel settings of multi-class classification and regression. First, in multi-class classification, a more general causal model is proposed under which the ISR-Multiclass algorithm is introduced: ISR-Multiclass can provably recover the invariant feature subspace in $\\lceil d_{spu}/k \\rceil + 1$ environments where $d_{spu}$ is the dimensionality of spurious features and $k$ is the number of classes. Thus, ISR-Multiclass leverages class information to \\textit{improve} the environment complexity by a factor of $k$ as compared to the original ISR-Mean, which requires $d_{spu} + 1$ environments. Next, in the setting of regression, ISR-Regression is introduced as a provable recovery algorithm under a new causal model for regression. ISR-Regression can identify the invariant feature subspace in $d_{spu} + 1$ environments, matching that of the original ISR algorithm. Empirically, ISR-Multiclass and ISR-Regression demonstrate superior performance across new synthetic linear benchmarks (in line with the theoretically claimed environment complexity) and significantly improve the robustness of neural networks trained with various methods (such as ERM, IRM) across synthetic and real-life datasets encoding strong spurious correlations - thus uncovering the generality, accuracy and efficiency of this framework."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Efficient invariant feature subspace recovery for domain generalization"]}]}],"canonical_facts":{"dc:contributor":["Zhao, Han"],"dc:creator":["Balasubramaniam, Gargi"],"dc:date":["2023-05","2023-04-24"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-09-01 without embargo terms","The student, Gargi Balasubramaniam, accepted the attached license on 2023-04-23 at 12:57.","The student, Gargi Balasubramaniam, submitted this Thesis for approval on 2023-04-23 at 13:04.","This Thesis was approved for publication on 2023-04-24 at 11:53.","DSpace SAF Submission Ingestion Package generated from Vireo submission #19121 on 2023-09-01 at 16:55:08","Domain generalization aims to learn a machine learning model over multiple training environments to generalize well to unseen test environments. With the advent of large pretrained models, recent work focuses on efficient domain generalization to maximize performance under minimal data and model fine-tuning requirements. Recently, Wang et al. proposed Invariant Feature Subspace Recovery (ISR-Mean), an efficient domain generalization algorithm which uses means of the class-conditional data distributions to provably identify the domain-invariant feature subspace under a given causal model. However, ISR-Mean is not applicable in the realistic setting of multi-class classification as it only utilizes information from a single class, failing to account for information across multiple classes. Further, it is unclear how this notion of invariant subspace recovery can be applied to regression, when there are no class labels. Motivated by the need to enable efficient robustness via invariant feature subspace recovery in these scenarios, this work extends ISR to two novel settings of multi-class classification and regression. First, in multi-class classification, a more general causal model is proposed under which the ISR-Multiclass algorithm is introduced: ISR-Multiclass can provably recover the invariant feature subspace in $\\lceil d_{spu}/k \\rceil + 1$ environments where $d_{spu}$ is the dimensionality of spurious features and $k$ is the number of classes. Thus, ISR-Multiclass leverages class information to \\textit{improve} the environment complexity by a factor of $k$ as compared to the original ISR-Mean, which requires $d_{spu} + 1$ environments. Next, in the setting of regression, ISR-Regression is introduced as a provable recovery algorithm under a new causal model for regression. ISR-Regression can identify the invariant feature subspace in $d_{spu} + 1$ environments, matching that of the original ISR algorithm. Empirically, ISR-Multiclass and ISR-Regression demonstrate superior performance across new synthetic linear benchmarks (in line with the theoretically claimed environment complexity) and significantly improve the robustness of neural networks trained with various methods (such as ERM, IRM) across synthetic and real-life datasets encoding strong spurious correlations - thus uncovering the generality, accuracy and efficiency of this framework."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/120084"],"dc:language":["en","eng"],"dc:rights":["Copyright 2023 Gargi Balasubramaniam"],"dc:subject":["Domain Generalization","Out-of-domain (OOD) Generalization","Invariant Feature Learning","Spurious Correlations"],"dc:title":["Efficient invariant feature subspace recovery for domain generalization"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:56Z"}