Efficient memory access in modern accelerators

This dissertation examines various methods of narrowing the performance gap between the main memory and processing units in accelerators. We divide approaches into two categories of improving the main memory system performance and reducing reliance on memory. Then, we offer novel techniques to address each group of approaches. To improve the performance of the memory system, first, we investigate near-DRAM acceleration. In this approach, the processing unit dies are stacked on top of dynamic random access memory (DRAM) dies through 3D die-stacking technology resulting in a power-efficient higher aggregate bandwidth memory system. Second, we propose in-buffer processing as an alternative to 3D die-stacking technology. In this solution, we place accelerators in data buffers of load-reduced dual inline memory module (LRDIMM) memory, which is originally developed to support large memory systems for servers, to avoid relying on 3D/2.5D die stacking. To reduce reliance on the memory system, we investigate various optimizations pertaining to sparse data orchestration. First, we propose hierarchical intersection to eliminate ineffectual data fetch and computation in sparse tensor algebra. Second, we investigate data orchestration, specifically dynamic tiling, in sparse matrix multiplication (SpMSpM) to reduce main memory traffic. Finally, we propose a generalized dynamic tiling and data orchestration mechanism for sparse tensor algebra. We present our generalized data orchestration unit as a generic primitive that can be integrated into accelerators with arbitrary dataflow, encompassing all the optimizations we proposed to reduce reliance on the memory system. This dissertation offers various mechanisms to overcome the memory access bottlenecks resulting in a holistic solution for efficient memory access in modern accelerators.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Asgharimoghaddam, Hadi

Contributors dc:contributor

• Fletcher, Christopher
• Emer, Joel
• Kumar, Rakesh
• Patel, Sanjay
• Solomonik, Edgar

• Computer Architecture
• Memory Systems
• 3D Die-Stacking
• Sparse Tensor Algebra
• Sparse Accelerators
• Dynamic Tiling

Statement dc:rights

• Copyright 2022 Hadi Asgharimoghaddam

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/115685

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F115685