{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/116250"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/116250","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Deep code: representation and learning algorithms for neural networks & their applications to communication codes","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2022-11-15 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2022-11-15 without embargo terms","abstract_has_math":false,"creators":["Makkuva, Ashok Vardhan"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Viswanath, Pramod","Hajek, Bruce","Rayadurgam, Srikant","Sun, Ruoyu","Oh, Sewoong"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2022,"date_issued":"2022-08","date_published":"2022-08","updated_at":"2026-07-22T22:24:55Z","subjects":["machine-learning","LSTM","GRU","mixture-of-experts","deep learning","KO codes","channel codes","error-correcting-codes","Reed-Muller codes","Polar codes"],"languages":["en","eng"],"rights":["Copyright 2022 Ashok Makkuva"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/116250","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Viswanath, Pramod","Hajek, Bruce","Rayadurgam, Srikant","Sun, Ruoyu","Oh, Sewoong"]},{"key":"dc:creator","label":"Author","values":["Makkuva, Ashok Vardhan"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2022-08","2022-07-15"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["machine-learning","LSTM","GRU","mixture-of-experts","deep learning","KO codes","channel codes","error-correcting-codes","Reed-Muller codes","Polar codes"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2022 Ashok Makkuva"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/116250"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2022-11-15 without embargo terms","The student, Ashok Makkuva, accepted the attached license on 2022-07-15 at 00:05.","The student, Ashok Makkuva, submitted this Dissertation for approval on 2022-07-15 at 00:10.","This Dissertation was approved for publication on 2022-07-15 at 12:22.","DSpace SAF Submission Ingestion Package generated from Vireo submission #18327 on 2022-11-15 at 18:21:15","Codes are the backbone of modern information age. Codes, composed of encoder and decoder pairs, are the basic mathematical objects that enable reliable communication. Landmark codes include convolutional, Reed-Muller, turbo, LDPC, and polar: each is linear and represents a mathematical breakthrough. Their impact on humanity is huge; each of these codes has been used in global communication standards over the past six decades. On the other hand, designing codes is a challenging task, mostly driven by human ingenuity. Befittingly, historically, the progress in discovery of codes has been sporadic. In this thesis, we present a new paradigm to invent codes via harnessing tools from deep-learning. Our major result is the invention of \\emph{KO codes}, a computationally efficient family of deep-learning driven codes that outperform the state-of-the-art RM and polar codes, in the challenging short-to-medium block length regime. The key technical innovation behind KO codes is the design of a novel family of neural architectures inspired by the computation tree of the {\\bf K}ronecker {\\bf O}peration (KO) central to RM and polar codes. These architectures pave the way for discovery of a much richer class of hitherto unexplored non-linear codes. This design technique can be viewed as an instantiation of the classical neural augmentation principle. In the process, we also study a popular neural network model called Mixture-of-Experts (MoE) that realizes this principle. We provide the first set of consistent and efficient algorithms with global learning guarantees for learning the parameters in a MoE which has been an open problem for more than two decades."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Deep code: representation and learning algorithms for neural networks & their applications to communication codes"]}]}],"canonical_facts":{"dc:contributor":["Viswanath, Pramod","Hajek, Bruce","Rayadurgam, Srikant","Sun, Ruoyu","Oh, Sewoong"],"dc:creator":["Makkuva, Ashok Vardhan"],"dc:date":["2022-08","2022-07-15"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2022-11-15 without embargo terms","The student, Ashok Makkuva, accepted the attached license on 2022-07-15 at 00:05.","The student, Ashok Makkuva, submitted this Dissertation for approval on 2022-07-15 at 00:10.","This Dissertation was approved for publication on 2022-07-15 at 12:22.","DSpace SAF Submission Ingestion Package generated from Vireo submission #18327 on 2022-11-15 at 18:21:15","Codes are the backbone of modern information age. Codes, composed of encoder and decoder pairs, are the basic mathematical objects that enable reliable communication. Landmark codes include convolutional, Reed-Muller, turbo, LDPC, and polar: each is linear and represents a mathematical breakthrough. Their impact on humanity is huge; each of these codes has been used in global communication standards over the past six decades. On the other hand, designing codes is a challenging task, mostly driven by human ingenuity. Befittingly, historically, the progress in discovery of codes has been sporadic. In this thesis, we present a new paradigm to invent codes via harnessing tools from deep-learning. Our major result is the invention of \\emph{KO codes}, a computationally efficient family of deep-learning driven codes that outperform the state-of-the-art RM and polar codes, in the challenging short-to-medium block length regime. The key technical innovation behind KO codes is the design of a novel family of neural architectures inspired by the computation tree of the {\\bf K}ronecker {\\bf O}peration (KO) central to RM and polar codes. These architectures pave the way for discovery of a much richer class of hitherto unexplored non-linear codes. This design technique can be viewed as an instantiation of the classical neural augmentation principle. In the process, we also study a popular neural network model called Mixture-of-Experts (MoE) that realizes this principle. We provide the first set of consistent and efficient algorithms with global learning guarantees for learning the parameters in a MoE which has been an open problem for more than two decades."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/116250"],"dc:language":["en","eng"],"dc:rights":["Copyright 2022 Ashok Makkuva"],"dc:subject":["machine-learning","LSTM","GRU","mixture-of-experts","deep learning","KO codes","channel codes","error-correcting-codes","Reed-Muller codes","Polar codes"],"dc:title":["Deep code: representation and learning algorithms for neural networks & their applications to communication codes"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:55Z"}