{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/107959"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/107959","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Universal and succinct source coding of deep neural networks","abstract":"Deep neural networks have shown incredible performance for inference tasks in a variety of domains. Unfortunately, most current deep networks are enormous cloud-based structures that require significant storage space, which limits scaling of deep learning as a service (DLaaS) and use for on-device intelligence. This work is concerned with finding universal lossless compressed representations of deep feedforward networks with synaptic weights drawn from discrete sets, and directly performing inference without full decompression. The basic insight that allows less rate than naive approaches is recognizing that the bipartite graph layers of feedforward networks have a kind of permutation invariance to the labeling of nodes, in terms of inferential operation. We provide efficient algorithms to dissipate this irrelevant uncertainty and then use arithmetic coding to nearly achieve the entropy bound in a universal manner. We also provide experimental results of our approach on several standard datasets.","abstract_html":"Deep neural networks have shown incredible performance for inference tasks in a variety of domains. Unfortunately, most current deep networks are enormous cloud-based structures that require significant storage space, which limits scaling of deep learning as a service (DLaaS) and use for on-device intelligence. This work is concerned with finding universal lossless compressed representations of deep feedforward networks with synaptic weights drawn from discrete sets, and directly performing inference without full decompression. The basic insight that allows less rate than naive approaches is recognizing that the bipartite graph layers of feedforward networks have a kind of permutation invariance to the labeling of nodes, in terms of inferential operation. We provide efficient algorithms to dissipate this irrelevant uncertainty and then use arithmetic coding to nearly achieve the entropy bound in a universal manner. We also provide experimental results of our approach on several standard datasets.","abstract_has_math":false,"creators":["Basu, Sourya"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Varshney, Lav R"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2020,"date_issued":"2020-08-26T21:54:44Z","date_published":"2020-08-26T21:54:44Z","updated_at":"2026-07-22T22:24:47Z","subjects":["Deep neural networks","Universal Source Coding"],"languages":["en"],"rights":["Copyright 2020 Sourya Basu"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/107959","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Varshney, Lav R"]},{"key":"dc:creator","label":"Author","values":["Basu, Sourya"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2020-08-26T21:54:44Z","2020-05-04","2020-05"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Deep neural networks","Universal Source Coding"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2020 Sourya Basu"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/107959"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Deep neural networks have shown incredible performance for inference tasks in a variety of domains. Unfortunately, most current deep networks are enormous cloud-based structures that require significant storage space, which limits scaling of deep learning as a service (DLaaS) and use for on-device intelligence. This work is concerned with finding universal lossless compressed representations of deep feedforward networks with synaptic weights drawn from discrete sets, and directly performing inference without full decompression. The basic insight that allows less rate than naive approaches is recognizing that the bipartite graph layers of feedforward networks have a kind of permutation invariance to the labeling of nodes, in terms of inferential operation. We provide efficient algorithms to dissipate this irrelevant uncertainty and then use arithmetic coding to nearly achieve the entropy bound in a universal manner. We also provide experimental results of our approach on several standard datasets.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2020-08-25 without embargo terms","The student, Sourya Basu, accepted the attached license on 2020-05-01 at 14:11.","The student, Sourya Basu, submitted this Thesis for approval on 2020-05-01 at 14:17.","This Thesis was approved for publication on 2020-05-04 at 16:24.","DSpace SAF Submission Ingestion Package generated from Vireo submission #15150 on 2020-08-25 at 17:10:29","Made available in DSpace on 2020-08-26T21:54:44Z (GMT). No. of bitstreams: 2 BASU-THESIS-2020.pdf: 367515 bytes, checksum: b06e61c5cfd3d2aa6f1420b685579b2c (MD5) LICENSE.txt: 4208 bytes, checksum: ea824d976212202d33d5229f749ca07b (MD5) Previous issue date: 2020-05-04"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Universal and succinct source coding of deep neural networks"]}]}],"canonical_facts":{"dc:contributor":["Varshney, Lav R"],"dc:creator":["Basu, Sourya"],"dc:date":["2020-08-26T21:54:44Z","2020-05-04","2020-05"],"dc:description":["Deep neural networks have shown incredible performance for inference tasks in a variety of domains. Unfortunately, most current deep networks are enormous cloud-based structures that require significant storage space, which limits scaling of deep learning as a service (DLaaS) and use for on-device intelligence. This work is concerned with finding universal lossless compressed representations of deep feedforward networks with synaptic weights drawn from discrete sets, and directly performing inference without full decompression. The basic insight that allows less rate than naive approaches is recognizing that the bipartite graph layers of feedforward networks have a kind of permutation invariance to the labeling of nodes, in terms of inferential operation. We provide efficient algorithms to dissipate this irrelevant uncertainty and then use arithmetic coding to nearly achieve the entropy bound in a universal manner. We also provide experimental results of our approach on several standard datasets.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2020-08-25 without embargo terms","The student, Sourya Basu, accepted the attached license on 2020-05-01 at 14:11.","The student, Sourya Basu, submitted this Thesis for approval on 2020-05-01 at 14:17.","This Thesis was approved for publication on 2020-05-04 at 16:24.","DSpace SAF Submission Ingestion Package generated from Vireo submission #15150 on 2020-08-25 at 17:10:29","Made available in DSpace on 2020-08-26T21:54:44Z (GMT). No. of bitstreams: 2 BASU-THESIS-2020.pdf: 367515 bytes, checksum: b06e61c5cfd3d2aa6f1420b685579b2c (MD5) LICENSE.txt: 4208 bytes, checksum: ea824d976212202d33d5229f749ca07b (MD5) Previous issue date: 2020-05-04"],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/107959"],"dc:language":["en"],"dc:rights":["Copyright 2020 Sourya Basu"],"dc:subject":["Deep neural networks","Universal Source Coding"],"dc:title":["Universal and succinct source coding of deep neural networks"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:47Z"}