Efficient inference of convolutional neural networks on general purpose hardware using weight repetition

Deep Neural Networks (DNNs) have begun to permeate all corners of electronic society due to their high accuracy and machine efficiency per operation. Recent work has shown how weights within and across DNN filters have large degrees of repetition due to the pigeonhole principle and modern weight quantization schemes, and that this weight repetition can be harnessed improve DNN inference efficiency in an accelerator/ASIC context. This thesis develops new techniques so that weight repetition leads to an efficiency gain on general-purpose and programmable SIMD-based architectures such as CPUs equipped with vector extensions. We show how to write high-performance software that does not require hardware modifications and can cope with the irregularity introduced by weight repetition schemes. Overall, our highly parallel software kernel achieves up to 1:51 speedup in runtime of inference over state-of-the-art baseline.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2019

Author dc:creator

• Agrawal, Rohit

Contributors dc:contributor

• Fletcher, Christopher W.

• Deep Neural Networks
• Convolutional Neural Networks
• Accelerator
• CPU
• GPU
• Deep Learning Hardware
• CNN Inference

Statement dc:rights

• Copyright 2019 Rohit Agrawal

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/105251

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/105251