MODEL ADAPTATION FOR EDGE AI

Deployment of Deep Neural Networks (DNNs) on edge devices presents significant challenges due to their computational demands. Existing model compression techniques often fall short by being oblivious to downstream user-specific tasks. This thesis addresses the challenge of adapting DNN models effectively on resource-limited hardware, advocating for flexible and efficient models. Firstly, we tackle the challenge of on-device adaptation for user-specific models. Given the limited on-chip memory in edge devices, data movement becomes a significant bottleneck, leading to increased energy consumption. We introduce Chameleon, an energy-efficient continual learning framework. Secondly, generic DNN models can lead to wasteful and inefficient inference. We propose CRISP, a class-aware pruning framework that employs a hybrid structured sparsity pattern. Thirdly, while low-precision integer formats are common for reduced precision arithmetic, these formats are inadequate for emerging workloads like transformers. We explore minifloats, reduced-precision floating-point formats that significantly reduce the memory footprint while offering greater flexibility due to their dynamic range. Finally, models such as Vision Transformers (ViTs) struggle with real-time inference on edge devices. We introduce SMS, a token pruning framework that dynamically determines which tokens to merge and to what extent, thereby optimizing model accuracy while adhering to the target average inference budget.

Author dc:creator

• SHIVAM AGGARWAL

• Quantization
• Sparsity
• Model Compression
• Efficient AI
• EdgeAI

Licence

• https://scholarbank.nus.edu.sg/bitstreams/aedc0f0c-f38b-4720-8a7f-e77860536d79/download