{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/121992"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/121992","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Reinforcement learning under general function approximation and novel interaction settings","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-03-01 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-03-01 without embargo terms","abstract_has_math":false,"creators":["Chen, Jinglin"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Jiang, Nan","Banerjee, Arindam","Raginsky, Maxim","Krishnamurthy, Akshay"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2023,"date_issued":"2023-12","date_published":"2023-12","updated_at":"2026-07-22T22:25:00Z","subjects":["Reinforcement Learning","Sample Complexity Analysis","Machine Learning","Artificial Intelligence"],"languages":["en","eng"],"rights":["Copyright 2023 Jinglin Chen"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/121992","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Jiang, Nan","Banerjee, Arindam","Raginsky, Maxim","Krishnamurthy, Akshay"]},{"key":"dc:creator","label":"Author","values":["Chen, Jinglin"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2023-12","2023-11-17"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Reinforcement Learning","Sample Complexity Analysis","Machine Learning","Artificial Intelligence"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2023 Jinglin Chen"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/121992"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-03-01 without embargo terms","The student, Jinglin Chen, accepted the attached license on 2023-11-15 at 18:15.","The student, Jinglin Chen, submitted this Dissertation for approval on 2023-11-15 at 18:31.","This Dissertation was approved for publication on 2023-11-17 at 11:43.","DSpace SAF Submission Ingestion Package generated from Vireo submission #19933 on 2024-03-01 at 13:14:34","Reinforcement Learning (RL) is an area of machine learning where an intelligent agent solves sequential decision-making problems based on experience. Recent advances in the applications of RL have been witnessed not only in the field of games, control systems, and healthcare, but also in language models and addressing open problems in mathematics. Despite the recent progress, RL algorithms are deemed to be data-hungry and unstable in training. Therefore, a flurry of research has focused on building fundamental guarantees for provably efficient RL algorithms. This line of research aims to provide a theoretical backbone and deepen our understanding of RL methods, which is crucial for the success of applications of RL in real-world scenarios. The main theme of this thesis is to propose and analyze sample efficient algorithms under different structural complexity measures and various interaction settings. In the thesis, we examine numerous setups: from pure offline RL to online RL with exploration, from Markov Decision Processes (MDPs) with low-rank structures to MDPs under more general function approximators, and from reward-aware to reward-free learning. More concretely, this thesis shows theoretical results in the sequel. The first part of this thesis revisits data coverage and function class representability assumptions in offline RL and makes an important conjecture that initiates future research. The second part provides a positive result in offline RL under non-exploratory data and weak function approximation. The third part focuses on the reward-free learning framework under general non-linear function approximation, bridges the sizeable gap in our understanding of reward-aware and reward-free settings, and shows an exponential separation between low-rank and linear completeness settings. Lastly, in the fourth part, we consider representation learning and reward-free learning in low-rank MDPs from the angle of the density candidate feature class. When establishing statistical complexity guarantees for RL algorithms under structural assumptions, we leverage powerful tools from machine learning theory literature. Throughout the thesis, we tackle complex scenarios in RL, where the agent faces challenges such as the absence of online data collection or a lack of reward information as guidance while exploring the environment. Overall, this thesis proposes sample efficient algorithms and presents theoretical results under general function approximation and novel interaction protocols."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Reinforcement learning under general function approximation and novel interaction settings"]}]}],"canonical_facts":{"dc:contributor":["Jiang, Nan","Banerjee, Arindam","Raginsky, Maxim","Krishnamurthy, Akshay"],"dc:creator":["Chen, Jinglin"],"dc:date":["2023-12","2023-11-17"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2024-03-01 without embargo terms","The student, Jinglin Chen, accepted the attached license on 2023-11-15 at 18:15.","The student, Jinglin Chen, submitted this Dissertation for approval on 2023-11-15 at 18:31.","This Dissertation was approved for publication on 2023-11-17 at 11:43.","DSpace SAF Submission Ingestion Package generated from Vireo submission #19933 on 2024-03-01 at 13:14:34","Reinforcement Learning (RL) is an area of machine learning where an intelligent agent solves sequential decision-making problems based on experience. Recent advances in the applications of RL have been witnessed not only in the field of games, control systems, and healthcare, but also in language models and addressing open problems in mathematics. Despite the recent progress, RL algorithms are deemed to be data-hungry and unstable in training. Therefore, a flurry of research has focused on building fundamental guarantees for provably efficient RL algorithms. This line of research aims to provide a theoretical backbone and deepen our understanding of RL methods, which is crucial for the success of applications of RL in real-world scenarios. The main theme of this thesis is to propose and analyze sample efficient algorithms under different structural complexity measures and various interaction settings. In the thesis, we examine numerous setups: from pure offline RL to online RL with exploration, from Markov Decision Processes (MDPs) with low-rank structures to MDPs under more general function approximators, and from reward-aware to reward-free learning. More concretely, this thesis shows theoretical results in the sequel. The first part of this thesis revisits data coverage and function class representability assumptions in offline RL and makes an important conjecture that initiates future research. The second part provides a positive result in offline RL under non-exploratory data and weak function approximation. The third part focuses on the reward-free learning framework under general non-linear function approximation, bridges the sizeable gap in our understanding of reward-aware and reward-free settings, and shows an exponential separation between low-rank and linear completeness settings. Lastly, in the fourth part, we consider representation learning and reward-free learning in low-rank MDPs from the angle of the density candidate feature class. When establishing statistical complexity guarantees for RL algorithms under structural assumptions, we leverage powerful tools from machine learning theory literature. Throughout the thesis, we tackle complex scenarios in RL, where the agent faces challenges such as the absence of online data collection or a lack of reward information as guidance while exploring the environment. Overall, this thesis proposes sample efficient algorithms and presents theoretical results under general function approximation and novel interaction protocols."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/121992"],"dc:language":["en","eng"],"dc:rights":["Copyright 2023 Jinglin Chen"],"dc:subject":["Reinforcement Learning","Sample Complexity Analysis","Machine Learning","Artificial Intelligence"],"dc:title":["Reinforcement learning under general function approximation and novel interaction settings"],"dc:type":["text"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:00Z"}