Acceleration of deep learning applications using Intel distribution of OpenVINO toolkit

Machine learning has been a popular domain of research for the past decade. The emergence of deep neural networks (DNN) brings new solutions to address complex problems, including image classification, object detection, and natural language processing (NLP). The use of convolution and deep architecture allows information to be extracted and learned effectively from large-scale datasets and has led to significant technological breakthroughs in many traditional scientific fields. In particular, astrophysicists have proposed deep learning solutions for tasks related to gravitational waves, such as detecting and characterizing such events. To satisfy the increasing demand from researchers, many open-source frameworks, such as TensorFlow and PyTorch have been developed for ease of use. With the assistance of large-scale distributed GPU systems, researchers are able to develop, train, and test domain-specific deep learning applications efficiently. However, many of such applications require deployment on edge devices, collecting and processing data in real-time. In this case, GPU may not provide a portable solution to DNN inference because they are expensive and power-inefficient. In contrast, other hardware architectures such as CPU, VPU, and FPGA can be more accessible to a larger range of customers and more applicable to many power-restricted scenarios. In this thesis, we are interested in accelerating DNN inference workload using Intel Distribution of OpenVINO toolkit on various Intel hardware. We explore the process of model conversion, workload deployment in Intel DevCloud, and performance benchmarks for several popular networks. In addition, we evaluate the inference performance of a specific deep learning algorithm developed for multi-messenger astrophysics to characterize complex gravitational waves caused by the merging of binary black holes. We make a comparative analysis in terms of inference throughput and power consumption on PyTorch with Nvidia GPUs and OpenVINO with Intel CPUs, GPUs, and VPUs.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Li, Haoxiang

Contributors dc:contributor

• Kindratenko, Volodymyr

• OpenVINO
• Machine Learning
• Convolution Neural Network
• Power-efficiency

Statement dc:rights

• Copyright 2022 Haoxiang Li

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/115631

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F115631