{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/115951"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/115951","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"A highly concurrent software-based sector cache for GPU orchestrated storage access","abstract":"This Thesis was approved for publication on 2022-07-18 at 17:02.","abstract_html":"This Thesis was approved for publication on 2022-07-18 at 17:02.","abstract_has_math":false,"creators":["Masood, Amna"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Hwu, Wen-mei W"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-08","date_published":"2022-08","updated_at":"2026-07-22T22:24:55Z","subjects":["Graphics Processing Unit","Storage","Cache"],"languages":["en","eng"],"rights":["Copyright 2022 Amna Masood"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/115951","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Hwu, Wen-mei W"]},{"key":"dc:creator","label":"Author","values":["Masood, Amna"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-08","2022-07-18"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Graphics Processing Unit","Storage","Cache"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2022 Amna Masood"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/115951"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["This Thesis was approved for publication on 2022-07-18 at 17:02.","DSpace SAF Submission Ingestion Package generated from Vireo submission #18364 on 2022-11-16 at 10:56:32","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-08-01","The student, Amna Masood, accepted the attached license on 2022-07-18 at 12:58.","The student, Amna Masood, submitted this Thesis for approval on 2022-07-18 at 13:25.","Heterogeneous systems with programmable accelerators are becoming increasingly popular with each passing day, as increasing demand for both data and computing power exceeds the capabilities of CPU-only systems. Graphics Processing Units (GPUs) are programmable accelerators widely used to meet the requirements of data and compute intensive applications, such as graph and data analytics, and recommender systems. However, these workloads require access to increasingly larger datasets, many of which cannot fit in GPU memory, hence limiting the portability of these applications to GPUs. In order to increase the effective memory capacity of GPUs, prior work Big Accelerator Memory (BaM) proposed to employ modern Solid State Storage devices as backup memory that GPU threads can access on-demand. To exploit the locality of the data and amortize the high storage access latency, BaM proposed a highly concurrent software-managed cache for GPUs. However, the BaM cache is not scalable for large dataset sizes, as it requires large metadata structures in GPU memory. The size of these structures is proportional to the size of the input dataset and exceeds the GPU memory capacity for larger datasets. To address these limitations, this thesis proposes a high-throughput, highly concurrent software-managed sector cache design. The proposed software-managed sector cache design has the same set of features as the BaM concurrent cache design but with significantly lower metadata size. This makes the proposed sector cache design highly scalable and a direct replacement for BaM cache design. Our proposed cache design provides high throughput and employs optimizations to reduce the number of I/O accesses and exploit temporal and spatial locality. We evaluate our cache design using off-the-shelf GPUs and commodity SSDs and observe that it can provide high throughput of up to 350 GB/s. Through micro-benchmarking with various access patterns, we observe that for same I/O access granularity (4KB) and same cacheline size (4KB), the additional atomics needed for sector state management sector incur only a 10-15% overhead in throughput over BaM’s baseline cache. As the sector cache separates the cache-line granularity from the I/O access granularity, it enables fine-grain access even with large cacheline sizes. When we increase the cacheline size for both sector and baseline cache to 32KB, keeping the sector size at 4KB, the sector cache outperforms the baseline cache by up to 87% for random sector access workloads, as the sector cache’s finer I/O access granularity causes less I/O amplification. For graph applications, sector cache’s performance with coarse cacheline sizes (32KB) is on-par with the baseline cache design, keeping the I/O granularity same for both (4KB), while reducing the metadata by up to 6x. This shows that sector cache can enable consistent or better performance while reducing the metadata size. These results show that sector cache is necessary to scale the baseline system to larger dataset sizes while maintaining consistent performance."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["A highly concurrent software-based sector cache for GPU orchestrated storage access"]}]}],"canonical_facts":{"dc:contributor":["Hwu, Wen-mei W"],"dc:creator":["Masood, Amna"],"dc:date":["2022-08","2022-07-18"],"dc:description":["This Thesis was approved for publication on 2022-07-18 at 17:02.","DSpace SAF Submission Ingestion Package generated from Vireo submission #18364 on 2022-11-16 at 10:56:32","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2024-08-01","The student, Amna Masood, accepted the attached license on 2022-07-18 at 12:58.","The student, Amna Masood, submitted this Thesis for approval on 2022-07-18 at 13:25.","Heterogeneous systems with programmable accelerators are becoming increasingly popular with each passing day, as increasing demand for both data and computing power exceeds the capabilities of CPU-only systems. Graphics Processing Units (GPUs) are programmable accelerators widely used to meet the requirements of data and compute intensive applications, such as graph and data analytics, and recommender systems. However, these workloads require access to increasingly larger datasets, many of which cannot fit in GPU memory, hence limiting the portability of these applications to GPUs. In order to increase the effective memory capacity of GPUs, prior work Big Accelerator Memory (BaM) proposed to employ modern Solid State Storage devices as backup memory that GPU threads can access on-demand. To exploit the locality of the data and amortize the high storage access latency, BaM proposed a highly concurrent software-managed cache for GPUs. However, the BaM cache is not scalable for large dataset sizes, as it requires large metadata structures in GPU memory. The size of these structures is proportional to the size of the input dataset and exceeds the GPU memory capacity for larger datasets. To address these limitations, this thesis proposes a high-throughput, highly concurrent software-managed sector cache design. The proposed software-managed sector cache design has the same set of features as the BaM concurrent cache design but with significantly lower metadata size. This makes the proposed sector cache design highly scalable and a direct replacement for BaM cache design. Our proposed cache design provides high throughput and employs optimizations to reduce the number of I/O accesses and exploit temporal and spatial locality. We evaluate our cache design using off-the-shelf GPUs and commodity SSDs and observe that it can provide high throughput of up to 350 GB/s. Through micro-benchmarking with various access patterns, we observe that for same I/O access granularity (4KB) and same cacheline size (4KB), the additional atomics needed for sector state management sector incur only a 10-15% overhead in throughput over BaM’s baseline cache. As the sector cache separates the cache-line granularity from the I/O access granularity, it enables fine-grain access even with large cacheline sizes. When we increase the cacheline size for both sector and baseline cache to 32KB, keeping the sector size at 4KB, the sector cache outperforms the baseline cache by up to 87% for random sector access workloads, as the sector cache’s finer I/O access granularity causes less I/O amplification. For graph applications, sector cache’s performance with coarse cacheline sizes (32KB) is on-par with the baseline cache design, keeping the I/O granularity same for both (4KB), while reducing the metadata by up to 6x. This shows that sector cache can enable consistent or better performance while reducing the metadata size. These results show that sector cache is necessary to scale the baseline system to larger dataset sizes while maintaining consistent performance."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/115951"],"dc:language":["en","eng"],"dc:rights":["Copyright 2022 Amna Masood"],"dc:subject":["Graphics Processing Unit","Storage","Cache"],"dc:title":["A highly concurrent software-based sector cache for GPU orchestrated storage access"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:55Z"}