{"id":{"repo_id":"mit","oai_identifier":"oai:dspace.mit.edu:1721.1/139138"},"canonical_url":"https://search.dev.ndltd.org/etd/mit/oai:dspace.mit.edu:1721.1/139138","repository":{"repo_id":"mit","name":"MIT","base_url":"https://dspace.mit.edu/oai/request"},"display":{"title":"Application-driven Intersections between Information Theory and Machine Learning","abstract":"Machine learning has been tremendously successful in the past decade. In this thesis, we introduce guidance and insights from information theory to practical machine learning algorithms. In particular, we study three application domains and demonstrate the algorithmic gain of integrating machine learning with information theory. In the first part of the thesis, we deploy the principle of network coding to propose a decomposition scheme for distributing a neural network over a physical communication network. We show through experiments that our proposed scheme dramatically reduces the energy used compared to existing communication schemes under various channel statistics and network topologies. In the second part, we design a learning-based coding scheme, developed from the concept of error correction codes, for bio-molecular profiling. We show through simulations that, with a learning-based encoder and a maximize a posterior (MAP) decoder, our scheme significantly outperforms existing schemes in reducing the false negative rate of rare bio-molecular types. In the third part, we exercise guesswork on the machine translation problem. We study machine translation using the seq2seq model and we provide insights into quantifying the uncertainty within. Our results shed light on the design of inference in machine translation for selecting the beam size in beam search.","abstract_html":"Machine learning has been tremendously successful in the past decade. In this thesis, we introduce guidance and insights from information theory to practical machine learning algorithms. In particular, we study three application domains and demonstrate the algorithmic gain of integrating machine learning with information theory. In the first part of the thesis, we deploy the principle of network coding to propose a decomposition scheme for distributing a neural network over a physical communication network. We show through experiments that our proposed scheme dramatically reduces the energy used compared to existing communication schemes under various channel statistics and network topologies. In the second part, we design a learning-based coding scheme, developed from the concept of error correction codes, for bio-molecular profiling. We show through simulations that, with a learning-based encoder and a maximize a posterior (MAP) decoder, our scheme significantly outperforms existing schemes in reducing the false negative rate of rare bio-molecular types. In the third part, we exercise guesswork on the machine translation problem. We study machine translation using the seq2seq model and we provide insights into quantifying the uncertainty within. Our results shed light on the design of inference in machine translation for selecting the beam size in beam search.","abstract_has_math":false,"creators":["Liu, Litian"],"institution":"Massachusetts Institute of Technology","degree_name":"Doctoral","degree_level":null,"degree_discipline":null,"degree_department":"Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science","school":null,"contributors":[],"advisors":["Médard, Muriel"],"committee_chairs":[],"committee_members":[],"year":2021,"date_issued":"2021-06","date_published":"2021-06","updated_at":"2026-07-22T22:21:36Z","subjects":[],"languages":[],"rights":["In Copyright - Educational Use Permitted","Copyright MIT"],"rights_urls":["http://rightsstatements.org/page/InC-EDU/1.0/"],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/1721.1/139138","outbound_label":"Handle","outbound_source":"dc:identifier.uri"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Médard, Muriel"]},{"key":"dc:contributor.department","label":"Department","values":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science"]},{"key":"dc:creator","label":"Author","values":["Liu, Litian"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2022-01-14T14:52:11Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2022-01-14T14:52:11Z"]},{"key":"dc:date.issued","label":"Date","values":["2021-06"]},{"key":"dc:publisher","label":"Institution","values":["Massachusetts Institute of Technology"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctoral","Doctor of Philosophy"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights","label":"Dc Rights","values":["In Copyright - Educational Use Permitted","Copyright MIT"]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://rightsstatements.org/page/InC-EDU/1.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/1721.1/139138"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Machine learning has been tremendously successful in the past decade. In this thesis, we introduce guidance and insights from information theory to practical machine learning algorithms. In particular, we study three application domains and demonstrate the algorithmic gain of integrating machine learning with information theory. In the first part of the thesis, we deploy the principle of network coding to propose a decomposition scheme for distributing a neural network over a physical communication network. We show through experiments that our proposed scheme dramatically reduces the energy used compared to existing communication schemes under various channel statistics and network topologies. In the second part, we design a learning-based coding scheme, developed from the concept of error correction codes, for bio-molecular profiling. We show through simulations that, with a learning-based encoder and a maximize a posterior (MAP) decoder, our scheme significantly outperforms existing schemes in reducing the false negative rate of rare bio-molecular types. In the third part, we exercise guesswork on the machine translation problem. We study machine translation using the seq2seq model and we provide insights into quantifying the uncertainty within. Our results shed light on the design of inference in machine translation for selecting the beam size in beam search."]},{"key":"dc:description.degree","label":"Dc Description Degree","values":["Ph.D."]},{"key":"dc:title","label":"Title","values":["Application-driven Intersections between Information Theory and Machine Learning"]}]}],"canonical_facts":{"dc:contributor.advisor":["Médard, Muriel"],"dc:contributor.department":["Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science"],"dc:creator":["Liu, Litian"],"dc:date.accessioned":["2022-01-14T14:52:11Z"],"dc:date.available":["2022-01-14T14:52:11Z"],"dc:date.issued":["2021-06"],"dc:description.abstract":["Machine learning has been tremendously successful in the past decade. In this thesis, we introduce guidance and insights from information theory to practical machine learning algorithms. In particular, we study three application domains and demonstrate the algorithmic gain of integrating machine learning with information theory. In the first part of the thesis, we deploy the principle of network coding to propose a decomposition scheme for distributing a neural network over a physical communication network. We show through experiments that our proposed scheme dramatically reduces the energy used compared to existing communication schemes under various channel statistics and network topologies. In the second part, we design a learning-based coding scheme, developed from the concept of error correction codes, for bio-molecular profiling. We show through simulations that, with a learning-based encoder and a maximize a posterior (MAP) decoder, our scheme significantly outperforms existing schemes in reducing the false negative rate of rare bio-molecular types. In the third part, we exercise guesswork on the machine translation problem. We study machine translation using the seq2seq model and we provide insights into quantifying the uncertainty within. Our results shed light on the design of inference in machine translation for selecting the beam size in beam search."],"dc:description.degree":["Ph.D."],"dc:identifier.uri":["https://hdl.handle.net/1721.1/139138"],"dc:publisher":["Massachusetts Institute of Technology"],"dc:rights":["In Copyright - Educational Use Permitted","Copyright MIT"],"dc:rights.uri":["http://rightsstatements.org/page/InC-EDU/1.0/"],"dc:title":["Application-driven Intersections between Information Theory and Machine Learning"],"dc:type":["Thesis"],"thesis:degree_name":["Doctoral","Doctor of Philosophy"]},"updated_at":"2026-07-22T22:21:36Z"}