{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/129914"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/129914","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Structure and representation in statistical reinforcement learning","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-20 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-20 without embargo terms","abstract_has_math":false,"creators":["Amortila, Philip"],"institution":"University of Illinois Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Jiang, Nan","Banerjee, Arindam","Raginsky, Maxim","Szepesvári, Csaba"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-07-02","date_published":"2025-07-02","updated_at":"2026-07-22T22:25:06Z","subjects":["Artificial Intelligence","Machine Learning","Reinforcement Learning","Statistics"],"languages":["en","eng"],"rights":["Copyright 2025 Philip Amortila"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/129914","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Jiang, Nan","Banerjee, Arindam","Raginsky, Maxim","Szepesvári, Csaba"]},{"key":"dc:creator","label":"Author","values":["Amortila, Philip"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-07-02","2025-08"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Artificial Intelligence","Machine Learning","Reinforcement Learning","Statistics"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2025 Philip Amortila"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/129914"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-20 without embargo terms","The student, Philip Amortila, accepted the attached license on 2025-06-30 at 18:40.","The student, Philip Amortila, submitted this Dissertation for approval on 2025-06-30 at 18:52.","This Dissertation was approved for publication on 2025-07-02 at 15:02.","DSpace SAF Submission Ingestion Package generated from Vireo submission #22389 on 2025-10-20 at 20:14:54","To what extent can advances in data-driven prediction be leveraged to improve data-driven decision-making? Reinforcement learning (RL) has been a promising approach for tackling automated sequential decision-making tasks, and in recent years has been instrumental for success in numerous impressive applications ranging from game-playing to recommendation systems to inventory management to the training of large generative models. These advances are largely powered by the merging of RL with modern function approximators (notably, neural networks). This thesis contributes to the statistical and algorithmic foundations of this framework. Our aim is to develop algorithms which leverage function approximation for scalability to large, complex sequential decision-making tasks. We seek methods with statistical complexities analogous to those obtained in simpler prediction tasks and algorithmically reduce to well-known primitives such as empirical risk minimization. Achieving this goal requires structural assumptions about the environment and/or representational assumptions about the function class. We investigate the nature of these assumptions in three parts. The first part of this thesis tackles the question of efficient RL with access to a simulator of the environment and linear features that satisfy only weak representational assumptions -- namely, which are only assumed to capture the optimal value function. In a series of three chapters, we establish that: 1) in general, sample-efficient RL is not possible under this minimal assumption, but that sample-efficiency is possible when 2) the action space is sufficiently small, or 3) a small amount of expert advice is available. The second part of this thesis examines the interaction between RL and function class misspecification. We demonstrate that, in RL, the error incurred by misspecification can be amplified due to the challenges of distribution shift and credit assignment. We study the optimal approximation factors incurred and develop algorithms which -- when possible -- mitigate this amplification. The third part of this thesis studies structural conditions which permit efficient online RL under general (possibly nonlinear) function approximation. We first leverage intimate connections between offline RL and online RL to derive algorithms that are statistically and computationally efficient under a general structural condition known as coverability. We then study the representation learning problem in settings where agents must make decisions from high-dimensional observations (such as images or sensor readings) that mask simpler underlying latent dynamics, with the goal of reducing problems to their latent complexity."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Structure and representation in statistical reinforcement learning"]}]}],"canonical_facts":{"dc:contributor":["Jiang, Nan","Banerjee, Arindam","Raginsky, Maxim","Szepesvári, Csaba"],"dc:creator":["Amortila, Philip"],"dc:date":["2025-07-02","2025-08"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-20 without embargo terms","The student, Philip Amortila, accepted the attached license on 2025-06-30 at 18:40.","The student, Philip Amortila, submitted this Dissertation for approval on 2025-06-30 at 18:52.","This Dissertation was approved for publication on 2025-07-02 at 15:02.","DSpace SAF Submission Ingestion Package generated from Vireo submission #22389 on 2025-10-20 at 20:14:54","To what extent can advances in data-driven prediction be leveraged to improve data-driven decision-making? Reinforcement learning (RL) has been a promising approach for tackling automated sequential decision-making tasks, and in recent years has been instrumental for success in numerous impressive applications ranging from game-playing to recommendation systems to inventory management to the training of large generative models. These advances are largely powered by the merging of RL with modern function approximators (notably, neural networks). This thesis contributes to the statistical and algorithmic foundations of this framework. Our aim is to develop algorithms which leverage function approximation for scalability to large, complex sequential decision-making tasks. We seek methods with statistical complexities analogous to those obtained in simpler prediction tasks and algorithmically reduce to well-known primitives such as empirical risk minimization. Achieving this goal requires structural assumptions about the environment and/or representational assumptions about the function class. We investigate the nature of these assumptions in three parts. The first part of this thesis tackles the question of efficient RL with access to a simulator of the environment and linear features that satisfy only weak representational assumptions -- namely, which are only assumed to capture the optimal value function. In a series of three chapters, we establish that: 1) in general, sample-efficient RL is not possible under this minimal assumption, but that sample-efficiency is possible when 2) the action space is sufficiently small, or 3) a small amount of expert advice is available. The second part of this thesis examines the interaction between RL and function class misspecification. We demonstrate that, in RL, the error incurred by misspecification can be amplified due to the challenges of distribution shift and credit assignment. We study the optimal approximation factors incurred and develop algorithms which -- when possible -- mitigate this amplification. The third part of this thesis studies structural conditions which permit efficient online RL under general (possibly nonlinear) function approximation. We first leverage intimate connections between offline RL and online RL to derive algorithms that are statistically and computationally efficient under a general structural condition known as coverability. We then study the representation learning problem in settings where agents must make decisions from high-dimensional observations (such as images or sensor readings) that mask simpler underlying latent dynamics, with the goal of reducing problems to their latent complexity."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/129914"],"dc:language":["en","eng"],"dc:rights":["Copyright 2025 Philip Amortila"],"dc:subject":["Artificial Intelligence","Machine Learning","Reinforcement Learning","Statistics"],"dc:title":["Structure and representation in statistical reinforcement learning"],"dc:type":["text"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:06Z"}