{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/108183"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/108183","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"An adaptive pruning algorithm for DNN compression","abstract":"In recent years, deep neural networks have achieved remarkable results in various artificial intelligence tasks such as image recognition and machine language translation. Although deep neural networks achieve state-of-the-art accuracy for various tasks, the high accuracy comes with high computational complexity and large network size which leads to over-parameterization. The computational complexity and large size of deep neural networks limit their applications for low-power and mobile platforms. Techniques such as pruning, quantization and binarization have been developed to reduce network parameter size. Pruning is the one of well-studied methods to solve network over-parameterization and reduce computational complexity. A typical pruning process includes three stages: train the network, remove redundant weights according to certain criteria, and fine-tune the reduced network. However, the three-stage method is time-consuming and frequently cannot fully compensate for the pruned neurons which are actually important. In this thesis, we propose a new algorithm which merges the removing stage and fine-tuning stage into the training stage. The new algorithm reduces time complexity of the pruning process and makes incorrectly pruned neurons recoverable without significant loss of accuracy. We also explore the effects of different pruning criteria calculation methods, weight updating methods and pruning rate changing methods on pruning algorithm performance.","abstract_html":"In recent years, deep neural networks have achieved remarkable results in various artificial intelligence tasks such as image recognition and machine language translation. Although deep neural networks achieve state-of-the-art accuracy for various tasks, the high accuracy comes with high computational complexity and large network size which leads to over-parameterization. The computational complexity and large size of deep neural networks limit their applications for low-power and mobile platforms. Techniques such as pruning, quantization and binarization have been developed to reduce network parameter size. Pruning is the one of well-studied methods to solve network over-parameterization and reduce computational complexity. A typical pruning process includes three stages: train the network, remove redundant weights according to certain criteria, and fine-tune the reduced network. However, the three-stage method is time-consuming and frequently cannot fully compensate for the pruned neurons which are actually important. In this thesis, we propose a new algorithm which merges the removing stage and fine-tuning stage into the training stage. The new algorithm reduces time complexity of the pruning process and makes incorrectly pruned neurons recoverable without significant loss of accuracy. We also explore the effects of different pruning criteria calculation methods, weight updating methods and pruning rate changing methods on pruning algorithm performance.","abstract_has_math":false,"creators":["Wu, Jiaying"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Kindratenko, Volodymyr"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2020,"date_issued":"2020-05","date_published":"2020-05","updated_at":"2026-07-22T22:24:47Z","subjects":["Pruning","DNN"],"languages":["en"],"rights":["Copyright 2020 Jiaying Wu"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/108183","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Kindratenko, Volodymyr"]},{"key":"dc:creator","label":"Author","values":["Wu, Jiaying"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2020-05","2020-08-26T23:58:46Z","2022-08-26T23:58:55Z","2020-05-12"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Pruning","DNN"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2020 Jiaying Wu"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/108183"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["In recent years, deep neural networks have achieved remarkable results in various artificial intelligence tasks such as image recognition and machine language translation. Although deep neural networks achieve state-of-the-art accuracy for various tasks, the high accuracy comes with high computational complexity and large network size which leads to over-parameterization. The computational complexity and large size of deep neural networks limit their applications for low-power and mobile platforms. Techniques such as pruning, quantization and binarization have been developed to reduce network parameter size. Pruning is the one of well-studied methods to solve network over-parameterization and reduce computational complexity. A typical pruning process includes three stages: train the network, remove redundant weights according to certain criteria, and fine-tune the reduced network. However, the three-stage method is time-consuming and frequently cannot fully compensate for the pruned neurons which are actually important. In this thesis, we propose a new algorithm which merges the removing stage and fine-tuning stage into the training stage. The new algorithm reduces time complexity of the pruning process and makes incorrectly pruned neurons recoverable without significant loss of accuracy. We also explore the effects of different pruning criteria calculation methods, weight updating methods and pruning rate changing methods on pruning algorithm performance.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2022-05-01","The student, Jiaying Wu, accepted the attached license on 2020-05-11 at 15:21.","The student, Jiaying Wu, submitted this Thesis for approval on 2020-05-11 at 15:29.","This Thesis was approved for publication on 2020-05-12 at 10:37.","DSpace SAF Submission Ingestion Package generated from Vireo submission #15331 on 2020-08-25 at 17:30:59","Made available in DSpace on 2020-08-26T23:58:46Z (GMT). No. of bitstreams: 2 WU-THESIS-2020.pdf: 1986305 bytes, checksum: aa1555ed8aafa177ca45fcf2fbecd293 (MD5) LICENSE.txt: 4207 bytes, checksum: ad6ae9e841403b939f60434a9a107b08 (MD5) Previous issue date: 2020-05-12","Embargo set by: Seth Robbins for item 115796 Lift date: 2022-08-26T23:58:55Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["An adaptive pruning algorithm for DNN compression"]}]}],"canonical_facts":{"dc:contributor":["Kindratenko, Volodymyr"],"dc:creator":["Wu, Jiaying"],"dc:date":["2020-05","2020-08-26T23:58:46Z","2022-08-26T23:58:55Z","2020-05-12"],"dc:description":["In recent years, deep neural networks have achieved remarkable results in various artificial intelligence tasks such as image recognition and machine language translation. Although deep neural networks achieve state-of-the-art accuracy for various tasks, the high accuracy comes with high computational complexity and large network size which leads to over-parameterization. The computational complexity and large size of deep neural networks limit their applications for low-power and mobile platforms. Techniques such as pruning, quantization and binarization have been developed to reduce network parameter size. Pruning is the one of well-studied methods to solve network over-parameterization and reduce computational complexity. A typical pruning process includes three stages: train the network, remove redundant weights according to certain criteria, and fine-tune the reduced network. However, the three-stage method is time-consuming and frequently cannot fully compensate for the pruned neurons which are actually important. In this thesis, we propose a new algorithm which merges the removing stage and fine-tuning stage into the training stage. The new algorithm reduces time complexity of the pruning process and makes incorrectly pruned neurons recoverable without significant loss of accuracy. We also explore the effects of different pruning criteria calculation methods, weight updating methods and pruning rate changing methods on pruning algorithm performance.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2022-05-01","The student, Jiaying Wu, accepted the attached license on 2020-05-11 at 15:21.","The student, Jiaying Wu, submitted this Thesis for approval on 2020-05-11 at 15:29.","This Thesis was approved for publication on 2020-05-12 at 10:37.","DSpace SAF Submission Ingestion Package generated from Vireo submission #15331 on 2020-08-25 at 17:30:59","Made available in DSpace on 2020-08-26T23:58:46Z (GMT). No. of bitstreams: 2 WU-THESIS-2020.pdf: 1986305 bytes, checksum: aa1555ed8aafa177ca45fcf2fbecd293 (MD5) LICENSE.txt: 4207 bytes, checksum: ad6ae9e841403b939f60434a9a107b08 (MD5) Previous issue date: 2020-05-12","Embargo set by: Seth Robbins for item 115796 Lift date: 2022-08-26T23:58:55Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only"],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/108183"],"dc:language":["en"],"dc:rights":["Copyright 2020 Jiaying Wu"],"dc:subject":["Pruning","DNN"],"dc:title":["An adaptive pruning algorithm for DNN compression"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:47Z"}