{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/132463"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/132463","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Learning-based control system design: theory and applications","abstract":"Reinforcement learning (RL) offers a versatile, data-driven framework for feedback controller synthesis applicable to a wide range of dynamical systems. Its adaptability makes RL suitable for large-scale, complex control applications where environments change rapidly and precise symbolic modeling is impractical. Despite this potential, the deployment of RL in real-world control systems remains limited due to the catastrophic risks associated with control failures. This dissertation advances the application of RL for control by establishing its theoretical foundation and demonstrating its practical capabilities. The first half of the dissertation develops model-free policy gradient (PG) algorithms with proven efficacy and efficiency for addressing fundamental benchmarks in control theory. These include state-feedback linear-quadratic regulator in Chapter 2, two-player zero-sum linear-quadratic dynamic game and H-infinity robust control in Chapter 3, Kalman filtering and output-feedback linear-quadratic-Gaussian control in Chapter 4, and terminal-state minimax estimation in Chapter 5. The central theme across these works is the development of control-specific RL algorithms, rather than the analysis of generic, out-of-the-box RL methods. Inspired by the strong mathematical foundations of model-based solvers that underpin traditional control theory, our approach leverages the rich structural properties of each control task. This methodology bridges the gap between model-based and RL-based control theories, enabling strong performance guarantees for data-driven controllers. To advance RL-based controllers toward reliable real-world deployment, the second half of this dissertation focuses on practical learning-based control system designs. This effort begins in Chapter 6 with Controlgym, an open-source benchmark of large-scale, safety-critical control applications designed for the rigorous evaluation of RL algorithms on metrics such as stability, robustness, efficiency, and scalability. Leveraging this testbed, we develop two distinct control architectures. Chapter 7 proposes a hybrid control architecture for nonlinear partial differential equations, where a controller derived from a data-driven surrogate model is used to warm-start a model-free policy optimization stage. This fine-tuning step compensates for errors in the surrogate model, improving control performance while maintaining high computational efficiency. Chapter 8 introduces a Decision Transformer, which reframes the control problem as a sequence prediction task. The Decision Transformer architecture demonstrates notable zero-shot generalization and rapid adaptation to new control tasks with minimal data. The dissertation concludes in Chapter 9 with a summary of findings and a discussion of future research directions.","abstract_html":"Reinforcement learning (RL) offers a versatile, data-driven framework for feedback controller synthesis applicable to a wide range of dynamical systems. Its adaptability makes RL suitable for large-scale, complex control applications where environments change rapidly and precise symbolic modeling is impractical. Despite this potential, the deployment of RL in real-world control systems remains limited due to the catastrophic risks associated with control failures. This dissertation advances the application of RL for control by establishing its theoretical foundation and demonstrating its practical capabilities. The first half of the dissertation develops model-free policy gradient (PG) algorithms with proven efficacy and efficiency for addressing fundamental benchmarks in control theory. These include state-feedback linear-quadratic regulator in Chapter 2, two-player zero-sum linear-quadratic dynamic game and H-infinity robust control in Chapter 3, Kalman filtering and output-feedback linear-quadratic-Gaussian control in Chapter 4, and terminal-state minimax estimation in Chapter 5. The central theme across these works is the development of control-specific RL algorithms, rather than the analysis of generic, out-of-the-box RL methods. Inspired by the strong mathematical foundations of model-based solvers that underpin traditional control theory, our approach leverages the rich structural properties of each control task. This methodology bridges the gap between model-based and RL-based control theories, enabling strong performance guarantees for data-driven controllers. To advance RL-based controllers toward reliable real-world deployment, the second half of this dissertation focuses on practical learning-based control system designs. This effort begins in Chapter 6 with Controlgym, an open-source benchmark of large-scale, safety-critical control applications designed for the rigorous evaluation of RL algorithms on metrics such as stability, robustness, efficiency, and scalability. Leveraging this testbed, we develop two distinct control architectures. Chapter 7 proposes a hybrid control architecture for nonlinear partial differential equations, where a controller derived from a data-driven surrogate model is used to warm-start a model-free policy optimization stage. This fine-tuning step compensates for errors in the surrogate model, improving control performance while maintaining high computational efficiency. Chapter 8 introduces a Decision Transformer, which reframes the control problem as a sequence prediction task. The Decision Transformer architecture demonstrates notable zero-shot generalization and rapid adaptation to new control tasks with minimal data. The dissertation concludes in Chapter 9 with a summary of findings and a discussion of future research directions.","abstract_has_math":false,"creators":["Zhang, Xiangyuan"],"institution":"University of Illinois Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Başar, Tamer","Srikant, Rayadurgam","Dullerud, Geir","Mitra, Sayan"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-12","date_published":"2025-12","updated_at":"2026-07-22T22:25:07Z","subjects":["Control systems","reinforcement learning","dynamical systems","optimization"],"languages":["en"],"rights":["Copyright 2025 Xiangyuan Zhang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/132463","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Başar, Tamer","Srikant, Rayadurgam","Dullerud, Geir","Mitra, Sayan"]},{"key":"dc:creator","label":"Author","values":["Zhang, Xiangyuan"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-12","2025-09-11"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Control systems","reinforcement learning","dynamical systems","optimization"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2025 Xiangyuan Zhang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/132463"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Reinforcement learning (RL) offers a versatile, data-driven framework for feedback controller synthesis applicable to a wide range of dynamical systems. Its adaptability makes RL suitable for large-scale, complex control applications where environments change rapidly and precise symbolic modeling is impractical. Despite this potential, the deployment of RL in real-world control systems remains limited due to the catastrophic risks associated with control failures. This dissertation advances the application of RL for control by establishing its theoretical foundation and demonstrating its practical capabilities. The first half of the dissertation develops model-free policy gradient (PG) algorithms with proven efficacy and efficiency for addressing fundamental benchmarks in control theory. These include state-feedback linear-quadratic regulator in Chapter 2, two-player zero-sum linear-quadratic dynamic game and H-infinity robust control in Chapter 3, Kalman filtering and output-feedback linear-quadratic-Gaussian control in Chapter 4, and terminal-state minimax estimation in Chapter 5. The central theme across these works is the development of control-specific RL algorithms, rather than the analysis of generic, out-of-the-box RL methods. Inspired by the strong mathematical foundations of model-based solvers that underpin traditional control theory, our approach leverages the rich structural properties of each control task. This methodology bridges the gap between model-based and RL-based control theories, enabling strong performance guarantees for data-driven controllers. To advance RL-based controllers toward reliable real-world deployment, the second half of this dissertation focuses on practical learning-based control system designs. This effort begins in Chapter 6 with Controlgym, an open-source benchmark of large-scale, safety-critical control applications designed for the rigorous evaluation of RL algorithms on metrics such as stability, robustness, efficiency, and scalability. Leveraging this testbed, we develop two distinct control architectures. Chapter 7 proposes a hybrid control architecture for nonlinear partial differential equations, where a controller derived from a data-driven surrogate model is used to warm-start a model-free policy optimization stage. This fine-tuning step compensates for errors in the surrogate model, improving control performance while maintaining high computational efficiency. Chapter 8 introduces a Decision Transformer, which reframes the control problem as a sequence prediction task. The Decision Transformer architecture demonstrates notable zero-shot generalization and rapid adaptation to new control tasks with minimal data. The dissertation concludes in Chapter 9 with a summary of findings and a discussion of future research directions.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2026-02-19 without embargo terms","The student, Xiangyuan Zhang, accepted the attached license on 2025-09-09 at 21:49.","The student, Xiangyuan Zhang, submitted this Dissertation for approval on 2025-09-09 at 22:01.","This Dissertation was approved for publication on 2025-09-11 at 11:55.","DSpace SAF Submission Ingestion Package generated from Vireo submission #22796 on 2026-02-19 at 18:24:15"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Learning-based control system design: theory and applications"]}]}],"canonical_facts":{"dc:contributor":["Başar, Tamer","Srikant, Rayadurgam","Dullerud, Geir","Mitra, Sayan"],"dc:creator":["Zhang, Xiangyuan"],"dc:date":["2025-12","2025-09-11"],"dc:description":["Reinforcement learning (RL) offers a versatile, data-driven framework for feedback controller synthesis applicable to a wide range of dynamical systems. Its adaptability makes RL suitable for large-scale, complex control applications where environments change rapidly and precise symbolic modeling is impractical. Despite this potential, the deployment of RL in real-world control systems remains limited due to the catastrophic risks associated with control failures. This dissertation advances the application of RL for control by establishing its theoretical foundation and demonstrating its practical capabilities. The first half of the dissertation develops model-free policy gradient (PG) algorithms with proven efficacy and efficiency for addressing fundamental benchmarks in control theory. These include state-feedback linear-quadratic regulator in Chapter 2, two-player zero-sum linear-quadratic dynamic game and H-infinity robust control in Chapter 3, Kalman filtering and output-feedback linear-quadratic-Gaussian control in Chapter 4, and terminal-state minimax estimation in Chapter 5. The central theme across these works is the development of control-specific RL algorithms, rather than the analysis of generic, out-of-the-box RL methods. Inspired by the strong mathematical foundations of model-based solvers that underpin traditional control theory, our approach leverages the rich structural properties of each control task. This methodology bridges the gap between model-based and RL-based control theories, enabling strong performance guarantees for data-driven controllers. To advance RL-based controllers toward reliable real-world deployment, the second half of this dissertation focuses on practical learning-based control system designs. This effort begins in Chapter 6 with Controlgym, an open-source benchmark of large-scale, safety-critical control applications designed for the rigorous evaluation of RL algorithms on metrics such as stability, robustness, efficiency, and scalability. Leveraging this testbed, we develop two distinct control architectures. Chapter 7 proposes a hybrid control architecture for nonlinear partial differential equations, where a controller derived from a data-driven surrogate model is used to warm-start a model-free policy optimization stage. This fine-tuning step compensates for errors in the surrogate model, improving control performance while maintaining high computational efficiency. Chapter 8 introduces a Decision Transformer, which reframes the control problem as a sequence prediction task. The Decision Transformer architecture demonstrates notable zero-shot generalization and rapid adaptation to new control tasks with minimal data. The dissertation concludes in Chapter 9 with a summary of findings and a discussion of future research directions.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2026-02-19 without embargo terms","The student, Xiangyuan Zhang, accepted the attached license on 2025-09-09 at 21:49.","The student, Xiangyuan Zhang, submitted this Dissertation for approval on 2025-09-09 at 22:01.","This Dissertation was approved for publication on 2025-09-11 at 11:55.","DSpace SAF Submission Ingestion Package generated from Vireo submission #22796 on 2026-02-19 at 18:24:15"],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/132463"],"dc:language":["en"],"dc:rights":["Copyright 2025 Xiangyuan Zhang"],"dc:subject":["Control systems","reinforcement learning","dynamical systems","optimization"],"dc:title":["Learning-based control system design: theory and applications"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:07Z"}