{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/102858"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/102858","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Improving cache replacement policy using deep reinforcement learning","abstract":"This thesis explores the use of reinforcement learning approaches to improve replacement policies of caches. In today's internet, caches play a vital role in improving performance of data transfers and load speeds. From video streaming to information retrieval from databases, caches allow applications to function more quickly and efficiently. A cache's replacement policy plays a major role in determining the cache's effectiveness and performance. The replacement policy is an algorithm that chooses which piece of data in the cache should be evicted when the cache becomes full and new elements are requested. In computer systems today, most caches use simple heuristic-based policies. Currently used policies are effective but are still far from optimal. Using more optimal cache replacement policies could dramatically improve internet performance and reduce database costs for many industry-based companies. This research examines learning more optimal replacement policies using reinforcement learning. In reinforcement learning, an agent learns to take optimal actions given information about an environment and a reward signal. In this work, deep reinforcement learning algorithms are trained to learn optimal cache replacement policies using a simulated cache environment and database access traces. This research presents the idea of using index-based cache access histories as input data for the reinforcement learning algorithms instead of content-based input. Several approaches are explored including value-based algorithms and policy gradient algorithms. The work presented here also explores the idea of using imitation learning algorithms to mimic optimal cache replacement policies. The algorithms are tested on several different cache sizes and data access patterns to show that these learned policies can outperform currently used replacement policies in a variety of settings.","abstract_html":"This thesis explores the use of reinforcement learning approaches to improve replacement policies of caches. In today's internet, caches play a vital role in improving performance of data transfers and load speeds. From video streaming to information retrieval from databases, caches allow applications to function more quickly and efficiently. A cache's replacement policy plays a major role in determining the cache's effectiveness and performance. The replacement policy is an algorithm that chooses which piece of data in the cache should be evicted when the cache becomes full and new elements are requested. In computer systems today, most caches use simple heuristic-based policies. Currently used policies are effective but are still far from optimal. Using more optimal cache replacement policies could dramatically improve internet performance and reduce database costs for many industry-based companies. This research examines learning more optimal replacement policies using reinforcement learning. In reinforcement learning, an agent learns to take optimal actions given information about an environment and a reward signal. In this work, deep reinforcement learning algorithms are trained to learn optimal cache replacement policies using a simulated cache environment and database access traces. This research presents the idea of using index-based cache access histories as input data for the reinforcement learning algorithms instead of content-based input. Several approaches are explored including value-based algorithms and policy gradient algorithms. The work presented here also explores the idea of using imitation learning algorithms to mimic optimal cache replacement policies. The algorithms are tested on several different cache sizes and data access patterns to show that these learned policies can outperform currently used replacement policies in a variety of settings.","abstract_has_math":false,"creators":["Benson, Christopher Edward"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Peng, Jian"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2019,"date_issued":"2019-02-07T20:44:29Z","date_published":"2019-02-07T20:44:29Z","updated_at":"2026-07-22T22:24:42Z","subjects":["Reinforcement Learning","Machine Learning","Deep Learning"],"languages":["en"],"rights":["Copyright 2018 Christopher Benson"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/102858","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Peng, Jian"]},{"key":"dc:creator","label":"Author","values":["Benson, Christopher Edward"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2019-02-07T20:44:29Z","2021-02-08T10:15:22Z","2018-12-13","2018-12"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Reinforcement Learning","Machine Learning","Deep Learning"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2018 Christopher Benson"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/102858"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["This thesis explores the use of reinforcement learning approaches to improve replacement policies of caches. In today's internet, caches play a vital role in improving performance of data transfers and load speeds. From video streaming to information retrieval from databases, caches allow applications to function more quickly and efficiently. A cache's replacement policy plays a major role in determining the cache's effectiveness and performance. The replacement policy is an algorithm that chooses which piece of data in the cache should be evicted when the cache becomes full and new elements are requested. In computer systems today, most caches use simple heuristic-based policies. Currently used policies are effective but are still far from optimal. Using more optimal cache replacement policies could dramatically improve internet performance and reduce database costs for many industry-based companies. This research examines learning more optimal replacement policies using reinforcement learning. In reinforcement learning, an agent learns to take optimal actions given information about an environment and a reward signal. In this work, deep reinforcement learning algorithms are trained to learn optimal cache replacement policies using a simulated cache environment and database access traces. This research presents the idea of using index-based cache access histories as input data for the reinforcement learning algorithms instead of content-based input. Several approaches are explored including value-based algorithms and policy gradient algorithms. The work presented here also explores the idea of using imitation learning algorithms to mimic optimal cache replacement policies. The algorithms are tested on several different cache sizes and data access patterns to show that these learned policies can outperform currently used replacement policies in a variety of settings.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2020-12-01","The student, Christopher Benson, accepted the attached license on 2018-12-12 at 16:59.","The student, Christopher Benson, submitted this Thesis for approval on 2018-12-12 at 17:03.","This Thesis was approved for publication on 2018-12-13 at 09:02.","DSpace SAF Submission Ingestion Package generated from Vireo submission #13291 on 2019-02-07 at 14:23:26","Made available in DSpace on 2019-02-07T20:44:29Z (GMT). No. of bitstreams: 2 BENSON-THESIS-2018.pdf: 551020 bytes, checksum: 510abce31c66d45aed67f1addb85b02f (MD5) LICENSE.txt: 4215 bytes, checksum: 14cc94c5754590b09abad33948e24f6a (MD5) Previous issue date: 2018-12-13","Embargo set by: Seth Robbins for item 109884 Lift date: 2021-02-07T20:44:35Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only Restriction Lifted for Item 109884 on 2021-02-08T10:15:22Z."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Improving cache replacement policy using deep reinforcement learning"]}]}],"canonical_facts":{"dc:contributor":["Peng, Jian"],"dc:creator":["Benson, Christopher Edward"],"dc:date":["2019-02-07T20:44:29Z","2021-02-08T10:15:22Z","2018-12-13","2018-12"],"dc:description":["This thesis explores the use of reinforcement learning approaches to improve replacement policies of caches. In today's internet, caches play a vital role in improving performance of data transfers and load speeds. From video streaming to information retrieval from databases, caches allow applications to function more quickly and efficiently. A cache's replacement policy plays a major role in determining the cache's effectiveness and performance. The replacement policy is an algorithm that chooses which piece of data in the cache should be evicted when the cache becomes full and new elements are requested. In computer systems today, most caches use simple heuristic-based policies. Currently used policies are effective but are still far from optimal. Using more optimal cache replacement policies could dramatically improve internet performance and reduce database costs for many industry-based companies. This research examines learning more optimal replacement policies using reinforcement learning. In reinforcement learning, an agent learns to take optimal actions given information about an environment and a reward signal. In this work, deep reinforcement learning algorithms are trained to learn optimal cache replacement policies using a simulated cache environment and database access traces. This research presents the idea of using index-based cache access histories as input data for the reinforcement learning algorithms instead of content-based input. Several approaches are explored including value-based algorithms and policy gradient algorithms. The work presented here also explores the idea of using imitation learning algorithms to mimic optimal cache replacement policies. The algorithms are tested on several different cache sizes and data access patterns to show that these learned policies can outperform currently used replacement policies in a variety of settings.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2020-12-01","The student, Christopher Benson, accepted the attached license on 2018-12-12 at 16:59.","The student, Christopher Benson, submitted this Thesis for approval on 2018-12-12 at 17:03.","This Thesis was approved for publication on 2018-12-13 at 09:02.","DSpace SAF Submission Ingestion Package generated from Vireo submission #13291 on 2019-02-07 at 14:23:26","Made available in DSpace on 2019-02-07T20:44:29Z (GMT). No. of bitstreams: 2 BENSON-THESIS-2018.pdf: 551020 bytes, checksum: 510abce31c66d45aed67f1addb85b02f (MD5) LICENSE.txt: 4215 bytes, checksum: 14cc94c5754590b09abad33948e24f6a (MD5) Previous issue date: 2018-12-13","Embargo set by: Seth Robbins for item 109884 Lift date: 2021-02-07T20:44:35Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only Restriction Lifted for Item 109884 on 2021-02-08T10:15:22Z."],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/102858"],"dc:language":["en"],"dc:rights":["Copyright 2018 Christopher Benson"],"dc:subject":["Reinforcement Learning","Machine Learning","Deep Learning"],"dc:title":["Improving cache replacement policy using deep reinforcement learning"],"dc:type":["text"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:42Z"}