{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/113048"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/113048","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Improved worst-case regret bounds for randomized least-squares value iteration","abstract":"This work studies regret minimization with randomized value functions in reinforcement learning. In tabular finite-horizon Markov Decision Processes, we introduce a clipping variant of one classical Thompson Sampling (TS)-like algorithm, randomized least-squares value iteration (RLSVI). Our $\\tilde{\\mathrm{O}}(H^2S\\sqrt{AT})$ high-probability worst-case regret bound improves the previous sharpest worst-case regret bounds for RLSVI and matches the existing state-of-the-art worst-case TS-based regret bounds.","abstract_html":"This work studies regret minimization with randomized value functions in reinforcement learning. In tabular finite-horizon Markov Decision Processes, we introduce a clipping variant of one classical Thompson Sampling (TS)-like algorithm, randomized least-squares value iteration (RLSVI). Our <span class=\"etd-inline-math\">\\tilde{<span class=\"etd-inline-math-roman\">O</span>}(H<sup>2</sup>S\\sqrt{AT})</span> high-probability worst-case regret bound improves the previous sharpest worst-case regret bounds for RLSVI and matches the existing state-of-the-art worst-case TS-based regret bounds.","abstract_has_math":true,"creators":["Agrawal, Priyank"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Jiang, Nan"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-01-12T21:45:44Z","date_published":"2022-01-12T21:45:44Z","updated_at":"2026-07-22T22:24:52Z","subjects":["Reinforcement Learning","Exploration-Exploitation"],"languages":["en"],"rights":["Copyright 2021 Priyank Agrawal"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/113048","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Jiang, Nan"]},{"key":"dc:creator","label":"Author","values":["Agrawal, Priyank"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-01-12T21:45:44Z","2021-07-15","2021-08"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Reinforcement Learning","Exploration-Exploitation"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2021 Priyank Agrawal"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/113048"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["This work studies regret minimization with randomized value functions in reinforcement learning. In tabular finite-horizon Markov Decision Processes, we introduce a clipping variant of one classical Thompson Sampling (TS)-like algorithm, randomized least-squares value iteration (RLSVI). Our $\\tilde{\\mathrm{O}}(H^2S\\sqrt{AT})$ high-probability worst-case regret bound improves the previous sharpest worst-case regret bounds for RLSVI and matches the existing state-of-the-art worst-case TS-based regret bounds.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2022-01-12 without embargo terms","The student, Priyank Agrawal, accepted the attached license on 2021-07-14 at 20:46.","The student, Priyank Agrawal, submitted this Thesis for approval on 2021-07-14 at 20:59.","This Thesis was approved for publication on 2021-07-15 at 15:44.","DSpace SAF Submission Ingestion Package generated from Vireo submission #16946 on 2022-01-12 at 12:45:32","Made available in DSpace on 2022-01-12T21:45:44Z (GMT). No. of bitstreams: 2 AGRAWAL-THESIS-2021.pdf: 427442 bytes, checksum: bbc5e57adc02afd39317656f59724340 (MD5) LICENSE.txt: 4212 bytes, checksum: 330f6b569f4b36aad71e7f0c0f652d33 (MD5) Previous issue date: 2021-07-15"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Improved worst-case regret bounds for randomized least-squares value iteration"]}]}],"canonical_facts":{"dc:contributor":["Jiang, Nan"],"dc:creator":["Agrawal, Priyank"],"dc:date":["2022-01-12T21:45:44Z","2021-07-15","2021-08"],"dc:description":["This work studies regret minimization with randomized value functions in reinforcement learning. In tabular finite-horizon Markov Decision Processes, we introduce a clipping variant of one classical Thompson Sampling (TS)-like algorithm, randomized least-squares value iteration (RLSVI). Our $\\tilde{\\mathrm{O}}(H^2S\\sqrt{AT})$ high-probability worst-case regret bound improves the previous sharpest worst-case regret bounds for RLSVI and matches the existing state-of-the-art worst-case TS-based regret bounds.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2022-01-12 without embargo terms","The student, Priyank Agrawal, accepted the attached license on 2021-07-14 at 20:46.","The student, Priyank Agrawal, submitted this Thesis for approval on 2021-07-14 at 20:59.","This Thesis was approved for publication on 2021-07-15 at 15:44.","DSpace SAF Submission Ingestion Package generated from Vireo submission #16946 on 2022-01-12 at 12:45:32","Made available in DSpace on 2022-01-12T21:45:44Z (GMT). No. of bitstreams: 2 AGRAWAL-THESIS-2021.pdf: 427442 bytes, checksum: bbc5e57adc02afd39317656f59724340 (MD5) LICENSE.txt: 4212 bytes, checksum: 330f6b569f4b36aad71e7f0c0f652d33 (MD5) Previous issue date: 2021-07-15"],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/113048"],"dc:language":["en"],"dc:rights":["Copyright 2021 Priyank Agrawal"],"dc:subject":["Reinforcement Learning","Exploration-Exploitation"],"dc:title":["Improved worst-case regret bounds for randomized least-squares value iteration"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:52Z"}