Comparing Learned Representations Between Unpruned and Pruned Deep Convolutional Neural Networks

<p>While deep neural networks have shown impressive performance in computer vision tasks, natural language processing, and other domains, the sizes and inference times of these models can often prevent them from being used on resource-constrained systems. Furthermore, as these networks grow larger in size and complexity, it can become even harder to understand the learned representations of the input data that these networks form through training. These issues of growing network size, increasing complexity and runtime, and ambiguity in the understanding of internal representations serve as guiding points for this work.</p> <p>In this thesis, we create a neural network that is capable of predicting up to three path waypoints given an input image. This network will be used in conjunction with other networks to help guide an autonomous robotic vehicle. Since this neural network will be deployed to an embedded system, it is important that our network is efficient. As such, we use a network compression technique known as L1 norm pruning to reduce the size of the network and speed up the inference time, while retaining similar loss. Furthermore, we investigate the effects that pruning has on the internal learned representations of models by comparing unpruned and pruned network layers using projection weighted canonical correlation analysis (PWCCA). Our results show that for deep convolutional neural networks (CNN), PWCCA similarity scores between early convolutional layers start low and then gradually increase towards the final layers of the network, with some peaks in the intermediate layers. We also show that for our deep CNN, linear layers at the end of the network also exhibit very high similarity, serving to guide the dissimilar representations from intermediate convolutional layers to a common representation that yields similar network performance between unpruned and pruned networks.</p>

Name thesis:degree_name

MS in Computer Science

Discipline thesis:degree_discipline

Computer Science

Year dc:date.available

2022

Author dc:creator

• Mitchell, Parker

Contributors dc:contributor

• John Seng
• Computer Science
• College of Engineering

• Machine Learning
• Neural Networks
• Autonomous Navigation
• Pruning
• Optimization
• Artificial Intelligence and Robotics
• Other Computer Sciences

Identifier

10.15368/theses.2022.45

OAI identifier oai:identifier

oai:digitalcommons.calpoly.edu:theses-4057