{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/124371"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/124371","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"An exploration of advancing efficiency in learning-based systems","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-09-16 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-09-16 without embargo terms","abstract_has_math":false,"creators":["Wang, Yite"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Mechanical Engineering","degree_department":null,"school":null,"contributors":["Hovakimyan, Naira","Sun, Ruoyu","Etesami, Seyed","Li, Yingying"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-05","date_published":"2024-05","updated_at":"2026-07-22T22:25:00Z","subjects":["Efficient Learning","Optimization","Statistical Learning","Learning-based System","Data Science","Deep Learning"],"languages":["en","eng"],"rights":["Copyright 2024 Yite Wang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/124371","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Hovakimyan, Naira","Sun, Ruoyu","Etesami, Seyed","Li, Yingying"]},{"key":"dc:creator","label":"Author","values":["Wang, Yite"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2024-05","2024-04-24"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Mechanical Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Efficient Learning","Optimization","Statistical Learning","Learning-based System","Data Science","Deep Learning"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2024 Yite Wang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/124371"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-09-16 without embargo terms","The student, Yite Wang, accepted the attached license on 2024-04-22 at 16:53.","The student, Yite Wang, submitted this Dissertation for approval on 2024-04-22 at 17:13.","This Dissertation was approved for publication on 2024-04-24 at 15:19.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20553 on 2024-09-16 at 00:35:49","Over the past decade, learning-based systems like deep neural networks (DNNs) have demonstrated remarkable success across a range of applications, such as machine vision, natural language processing, computational physics, and robotics. However, these modern DNNs often have a large number of parameters, leading to high computational demands in both training and deployment. This issue is particularly acute in the context of large-scale models like the generative pretrained transformer, e.g., GPT-3, where training expenses can cost millions of dollars. Moreover, the deployment of these models in resource-constrained environments, such as mobile robotics, drones, and intelligent driving systems, further exacerbates the challenge, hindering the full exploitation of DNNs' potential. In my Ph.D. research, I have dedicated my efforts to addressing these computational burdens, focusing on both the training and inference stages. From large to small. The initial part of my work focuses on developing more efficient neural networks through neural architecture search (NAS) and pruning, tailored to various optimization problems. We start with the application for classical minimization problems, leveraging the Neural Tangent Kernel (NTK) theory in Chapter 3 to introduce a novel pruning-at-initialization method tailored for image classification tasks, thereby optimizing computational efficiency. Progressing to bi-level optimization problem, i.e., meta-learning ('learning to learn'), in chapter 4, we explore NAS to automate the process of finding efficient neural network architecture, based on the theoretical insights provided by the NTK theory. Finally, we study the min-max problem associated with generative adversarial neural networks (GANs) in chapter 5. We investigate pruning-during-training method to reduce both training and inference costs of sparse GANs. From small to large. The latter part of my research focuses on transferring knowledge from small networks to larger networks for efficient training. In chapter 6, we present a novel approach to initialize large neural networks by leveraging pre-trained smaller networks, thereby substantially reducing the computational demands of the training process. The effectiveness of these methodologies is empirically demonstrated across a spectrum of applications. Our proposed methods have shown remarkable performance in tasks such as image generation, classification, and language understanding, setting new benchmarks for computational efficiency. Overall, this body of work not only contributes to the practicality and sustainability of DNNs in resource-constrained scenarios but also hopes to inspire future advancements in the field, paving the way for more efficient and accessible learning-based systems."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["An exploration of advancing efficiency in learning-based systems"]}]}],"canonical_facts":{"dc:contributor":["Hovakimyan, Naira","Sun, Ruoyu","Etesami, Seyed","Li, Yingying"],"dc:creator":["Wang, Yite"],"dc:date":["2024-05","2024-04-24"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-09-16 without embargo terms","The student, Yite Wang, accepted the attached license on 2024-04-22 at 16:53.","The student, Yite Wang, submitted this Dissertation for approval on 2024-04-22 at 17:13.","This Dissertation was approved for publication on 2024-04-24 at 15:19.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20553 on 2024-09-16 at 00:35:49","Over the past decade, learning-based systems like deep neural networks (DNNs) have demonstrated remarkable success across a range of applications, such as machine vision, natural language processing, computational physics, and robotics. However, these modern DNNs often have a large number of parameters, leading to high computational demands in both training and deployment. This issue is particularly acute in the context of large-scale models like the generative pretrained transformer, e.g., GPT-3, where training expenses can cost millions of dollars. Moreover, the deployment of these models in resource-constrained environments, such as mobile robotics, drones, and intelligent driving systems, further exacerbates the challenge, hindering the full exploitation of DNNs' potential. In my Ph.D. research, I have dedicated my efforts to addressing these computational burdens, focusing on both the training and inference stages. From large to small. The initial part of my work focuses on developing more efficient neural networks through neural architecture search (NAS) and pruning, tailored to various optimization problems. We start with the application for classical minimization problems, leveraging the Neural Tangent Kernel (NTK) theory in Chapter 3 to introduce a novel pruning-at-initialization method tailored for image classification tasks, thereby optimizing computational efficiency. Progressing to bi-level optimization problem, i.e., meta-learning ('learning to learn'), in chapter 4, we explore NAS to automate the process of finding efficient neural network architecture, based on the theoretical insights provided by the NTK theory. Finally, we study the min-max problem associated with generative adversarial neural networks (GANs) in chapter 5. We investigate pruning-during-training method to reduce both training and inference costs of sparse GANs. From small to large. The latter part of my research focuses on transferring knowledge from small networks to larger networks for efficient training. In chapter 6, we present a novel approach to initialize large neural networks by leveraging pre-trained smaller networks, thereby substantially reducing the computational demands of the training process. The effectiveness of these methodologies is empirically demonstrated across a spectrum of applications. Our proposed methods have shown remarkable performance in tasks such as image generation, classification, and language understanding, setting new benchmarks for computational efficiency. Overall, this body of work not only contributes to the practicality and sustainability of DNNs in resource-constrained scenarios but also hopes to inspire future advancements in the field, paving the way for more efficient and accessible learning-based systems."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/124371"],"dc:language":["en","eng"],"dc:rights":["Copyright 2024 Yite Wang"],"dc:subject":["Efficient Learning","Optimization","Statistical Learning","Learning-based System","Data Science","Deep Learning"],"dc:title":["An exploration of advancing efficiency in learning-based systems"],"dc:type":["text"],"thesis:degree_discipline":["Mechanical Engineering"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:00Z"}