{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/127191"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/127191","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Bayesian sparsity learning with variational automatic relevance determination","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-03-28 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-03-28 without embargo terms","abstract_has_math":false,"creators":["Liu, Zihe"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Statistics","degree_department":null,"school":null,"contributors":["Liu, Jingbo","Yang, Yun","Liang, Feng","Chen, Yuguo"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-12","date_published":"2024-12","updated_at":"2026-07-22T22:25:03Z","subjects":["Sparsity","Automatic Relevance Determination","Em- Pirical Bayes","Variational Technique","High-dimensional Linear Regression","Generalized Additive Model","Convergence"],"languages":["en","eng"],"rights":["Copyright 2024 Zihe Liu"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/127191","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Liu, Jingbo","Yang, Yun","Liang, Feng","Chen, Yuguo"]},{"key":"dc:creator","label":"Author","values":["Liu, Zihe"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2024-12","2024-11-18"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Statistics"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Sparsity","Automatic Relevance Determination","Em- Pirical Bayes","Variational Technique","High-dimensional Linear Regression","Generalized Additive Model","Convergence"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2024 Zihe Liu"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/127191"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-03-28 without embargo terms","The student, Zihe Liu, accepted the attached license on 2024-11-14 at 17:18.","The student, Zihe Liu, submitted this Dissertation for approval on 2024-11-14 at 17:31.","This Dissertation was approved for publication on 2024-11-18 at 14:47.","DSpace SAF Submission Ingestion Package generated from Vireo submission #21321 on 2025-03-28 at 14:25:38","Automatic Relevance Determination (ARD) is a well-regarded Bayesian approach for feature selection, where each feature’s relevance is encoded in a hyper-parameter that is automatically tuned through the data. However, estimating the ARD prior via the evidence function poses significant computational challenges, with no closed-form solution and scalability issues. Existing ARD research primarily focuses on algorithm development, with limited theoretical understanding of its properties. In this thesis, we introduce Variational Automatic Relevance Determination (VARD), a novel approach that estimates the ARD prior efficiently through a variational method. We examine the statistical properties of VARD in the context of high-dimensional linear regression, providing convergence guarantees for both parameter estimation and variable selection. Additionally, we extend the VARD framework to additive models, enabling simultaneous estimation of smoothness and relevance for each feature. The first part of this thesis studies the ARD procedure within high-dimensional linear regression under sparsity assumptions. Our proposed VARD method approximates the posterior distribution with independent Gaussian distributions for each regression coefficient, where some distributions converge to a point mass at zero, automatically excluding irrelevant variables. We establish convergence results and present an efficient coordinate descent algorithm to implement VARD, demonstrating its empirical performance on simulated datasets. In the second part, we extend VARD to sparse additive models in high-dimensional settings. VARD uniquely enables independent smoothness estimation for each feature, distinguishing whether a feature’s effect on the response is zero, linear, or nonlinear. An efficient coordinate descent algorithm further supports this implementation. Empirical evaluations on simulated and real-world data highlight VARD’s advantages over alternative variable selection methods for additive models."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Bayesian sparsity learning with variational automatic relevance determination"]}]}],"canonical_facts":{"dc:contributor":["Liu, Jingbo","Yang, Yun","Liang, Feng","Chen, Yuguo"],"dc:creator":["Liu, Zihe"],"dc:date":["2024-12","2024-11-18"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-03-28 without embargo terms","The student, Zihe Liu, accepted the attached license on 2024-11-14 at 17:18.","The student, Zihe Liu, submitted this Dissertation for approval on 2024-11-14 at 17:31.","This Dissertation was approved for publication on 2024-11-18 at 14:47.","DSpace SAF Submission Ingestion Package generated from Vireo submission #21321 on 2025-03-28 at 14:25:38","Automatic Relevance Determination (ARD) is a well-regarded Bayesian approach for feature selection, where each feature’s relevance is encoded in a hyper-parameter that is automatically tuned through the data. However, estimating the ARD prior via the evidence function poses significant computational challenges, with no closed-form solution and scalability issues. Existing ARD research primarily focuses on algorithm development, with limited theoretical understanding of its properties. In this thesis, we introduce Variational Automatic Relevance Determination (VARD), a novel approach that estimates the ARD prior efficiently through a variational method. We examine the statistical properties of VARD in the context of high-dimensional linear regression, providing convergence guarantees for both parameter estimation and variable selection. Additionally, we extend the VARD framework to additive models, enabling simultaneous estimation of smoothness and relevance for each feature. The first part of this thesis studies the ARD procedure within high-dimensional linear regression under sparsity assumptions. Our proposed VARD method approximates the posterior distribution with independent Gaussian distributions for each regression coefficient, where some distributions converge to a point mass at zero, automatically excluding irrelevant variables. We establish convergence results and present an efficient coordinate descent algorithm to implement VARD, demonstrating its empirical performance on simulated datasets. In the second part, we extend VARD to sparse additive models in high-dimensional settings. VARD uniquely enables independent smoothness estimation for each feature, distinguishing whether a feature’s effect on the response is zero, linear, or nonlinear. An efficient coordinate descent algorithm further supports this implementation. Empirical evaluations on simulated and real-world data highlight VARD’s advantages over alternative variable selection methods for additive models."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/127191"],"dc:language":["en","eng"],"dc:rights":["Copyright 2024 Zihe Liu"],"dc:subject":["Sparsity","Automatic Relevance Determination","Em- Pirical Bayes","Variational Technique","High-dimensional Linear Regression","Generalized Additive Model","Convergence"],"dc:title":["Bayesian sparsity learning with variational automatic relevance determination"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Statistics"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:03Z"}