{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/120125"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/120125","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Accelerating deep learning in the post-Moore’s era of computing","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-09-01 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-09-01 without embargo terms","abstract_has_math":false,"creators":["Hegde, Kartik"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Fletcher, Christopher W","Adve, Sarita","Hwu, Wen-mei","Pellauer, Michael","Torellas, Josep"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2023,"date_issued":"2023-05","date_published":"2023-05","updated_at":"2026-07-22T22:24:56Z","subjects":["Deep Learning","Computer Architecture","Moore's Law","Specialized Accelerators","Sparsity","Deep Neural Networks","Programmable Accelerators"],"languages":["en","eng"],"rights":["Copyright 2023 Kartik Hegde"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/120125","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Fletcher, Christopher W","Adve, Sarita","Hwu, Wen-mei","Pellauer, Michael","Torellas, Josep"]},{"key":"dc:creator","label":"Author","values":["Hegde, Kartik"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2023-05","2023-04-28"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Deep Learning","Computer Architecture","Moore's Law","Specialized Accelerators","Sparsity","Deep Neural Networks","Programmable Accelerators"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2023 Kartik Hegde"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/120125"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-09-01 without embargo terms","The student, Kartik Hegde, accepted the attached license on 2023-04-27 at 13:03.","The student, Kartik Hegde, submitted this Dissertation for approval on 2023-04-27 at 13:09.","This Dissertation was approved for publication on 2023-04-28 at 09:05.","DSpace SAF Submission Ingestion Package generated from Vireo submission #19213 on 2023-09-01 at 16:55:42","Rapid growth in the availability of cheap and fast computing power—made possible by Moore’s scaling—has been a major tailwind behind the rise of modern deep learning. However, over the last decade, the performance deficit between compute requirements of deep learning workloads and the peak compute capability offered by modern processors has been rapidly increasing due to two opposing trends. On the one hand, improving the accuracy of deep learning models requires larger models and datasets, which in turn requires more compute resources. On the other hand, Moore’s scaling has been slowing down, leading to a slowdown in the rate of increase in peak compute throughput. While specialization has been seen as a way to address the increasing performance deficit, there is mounting evidence that specialization has only afforded us a “one-time” boost in performance. To make matters worse, the increasing gap between logic and memory, often called the memory wall, makes it harder to utilize all the compute on-chip, let alone add more. Therefore, the key question facing deep learning architects is, how do we continue to accelerate deep learning workloads in the post-Moore’s era of computing? This thesis presents a set of techniques to accelerate deep learning workloads without scaling the number of transistors on-chip. At a high level, we classify them into two thrusts. First, techniques that reduce the total compute required for deep learning workloads without sacrificing accuracy and utilize that to improve performance. Second, techniques that increase the compute-per-transistor by improving the resource utilization of deep learning accelerators. Notably, we show that the proposed techniques are not “zero-sum”, i.e., the gained performance comes at negligible area/power overheads. We begin by describing a baseline deep learning accelerator (DLA) tailored for deep learning workloads and represents state-of-the-art deep learning accelerators. We then describe concrete ways to evolve the baseline architecture to incorporate our proposed techniques. Finally, we provide a detailed analysis of the area overheads of the proposed techniques and demonstrate that they are minimal while the performance gains are significant. Overall, this thesis throws light on different directions that computer architects can take to continue to accelerate deep learning workloads without scaling the number of transistors on-chip. We believe that expanding the scope of specialization beyond hardware to include other layers of the deep learning stack and careful co-design will enable us to continue to accelerate deep learning workloads in the post-Moore’s era of computing."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Accelerating deep learning in the post-Moore’s era of computing"]}]}],"canonical_facts":{"dc:contributor":["Fletcher, Christopher W","Adve, Sarita","Hwu, Wen-mei","Pellauer, Michael","Torellas, Josep"],"dc:creator":["Hegde, Kartik"],"dc:date":["2023-05","2023-04-28"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2023-09-01 without embargo terms","The student, Kartik Hegde, accepted the attached license on 2023-04-27 at 13:03.","The student, Kartik Hegde, submitted this Dissertation for approval on 2023-04-27 at 13:09.","This Dissertation was approved for publication on 2023-04-28 at 09:05.","DSpace SAF Submission Ingestion Package generated from Vireo submission #19213 on 2023-09-01 at 16:55:42","Rapid growth in the availability of cheap and fast computing power—made possible by Moore’s scaling—has been a major tailwind behind the rise of modern deep learning. However, over the last decade, the performance deficit between compute requirements of deep learning workloads and the peak compute capability offered by modern processors has been rapidly increasing due to two opposing trends. On the one hand, improving the accuracy of deep learning models requires larger models and datasets, which in turn requires more compute resources. On the other hand, Moore’s scaling has been slowing down, leading to a slowdown in the rate of increase in peak compute throughput. While specialization has been seen as a way to address the increasing performance deficit, there is mounting evidence that specialization has only afforded us a “one-time” boost in performance. To make matters worse, the increasing gap between logic and memory, often called the memory wall, makes it harder to utilize all the compute on-chip, let alone add more. Therefore, the key question facing deep learning architects is, how do we continue to accelerate deep learning workloads in the post-Moore’s era of computing? This thesis presents a set of techniques to accelerate deep learning workloads without scaling the number of transistors on-chip. At a high level, we classify them into two thrusts. First, techniques that reduce the total compute required for deep learning workloads without sacrificing accuracy and utilize that to improve performance. Second, techniques that increase the compute-per-transistor by improving the resource utilization of deep learning accelerators. Notably, we show that the proposed techniques are not “zero-sum”, i.e., the gained performance comes at negligible area/power overheads. We begin by describing a baseline deep learning accelerator (DLA) tailored for deep learning workloads and represents state-of-the-art deep learning accelerators. We then describe concrete ways to evolve the baseline architecture to incorporate our proposed techniques. Finally, we provide a detailed analysis of the area overheads of the proposed techniques and demonstrate that they are minimal while the performance gains are significant. Overall, this thesis throws light on different directions that computer architects can take to continue to accelerate deep learning workloads without scaling the number of transistors on-chip. We believe that expanding the scope of specialization beyond hardware to include other layers of the deep learning stack and careful co-design will enable us to continue to accelerate deep learning workloads in the post-Moore’s era of computing."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/120125"],"dc:language":["en","eng"],"dc:rights":["Copyright 2023 Kartik Hegde"],"dc:subject":["Deep Learning","Computer Architecture","Moore's Law","Specialized Accelerators","Sparsity","Deep Neural Networks","Programmable Accelerators"],"dc:title":["Accelerating deep learning in the post-Moore’s era of computing"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:56Z"}