{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/15989"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/15989","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Estimating directed information to infer causal relationships between neural spike trains and approximating discrete probability distributions with causal dependence trees","abstract":"\"This work examines an information theoretic quantity known as directed information, which measures statistically causal influences between processes. It is shown to be a general quantity, applicable to arbitrary probability distributions. It is interpreted in terms of prediction, communication with feedback, source coding with feed forward, control over noisy channels, and other settings. It is also shown to be consistent with Granger's philosophical definition. The concepts of direct and indirect causation in a network of processes are formalized. Next, two applications of directed information are investigated. Neuroscience researchers have been attempting to identify causal relationships between neural spike trains in electrode recordings, but have been doing so with correlation measures and measures based on Granger causality. We discuss why these methods are not robust, and do not have statistical guarantees. We use a point process GLM model and MDL (as a model order selection tool) for consistent estimation of directed information between neural spike trains. We have successfully applied this methodology to a network of simulated neurons and electrode array recordings. This work then develops a procedure, similar to Chow and Liu's, for fi nding the \"\"best\"\" approximation (in terms of KL divergence) of a full, joint distribution over a set of random processes, using a causal dependence tree distribution. Chow and Liu's procedure had been shown to be equivalent to maximizing a sum of mutual informations, and the procedure presented here is shown to be equivalent to maximizing a sum of directed informations. An algorithm is presented for efficiently finding the optimal causal tree, similar to that in Chow and Liu's work.\"","abstract_html":""This work examines an information theoretic quantity known as directed information, which measures statistically causal influences between processes. It is shown to be a general quantity, applicable to arbitrary probability distributions. It is interpreted in terms of prediction, communication with feedback, source coding with feed forward, control over noisy channels, and other settings. It is also shown to be consistent with Granger's philosophical definition. The concepts of direct and indirect causation in a network of processes are formalized. Next, two applications of directed information are investigated. Neuroscience researchers have been attempting to identify causal relationships between neural spike trains in electrode recordings, but have been doing so with correlation measures and measures based on Granger causality. We discuss why these methods are not robust, and do not have statistical guarantees. We use a point process GLM model and MDL (as a model order selection tool) for consistent estimation of directed information between neural spike trains. We have successfully applied this methodology to a network of simulated neurons and electrode array recordings. This work then develops a procedure, similar to Chow and Liu's, for fi nding the ""best"" approximation (in terms of KL divergence) of a full, joint distribution over a set of random processes, using a causal dependence tree distribution. Chow and Liu's procedure had been shown to be equivalent to maximizing a sum of mutual informations, and the procedure presented here is shown to be equivalent to maximizing a sum of directed informations. An algorithm is presented for efficiently finding the optimal causal tree, similar to that in Chow and Liu's work."","abstract_has_math":false,"creators":["Quinn, Christopher J."],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Coleman, Todd P.","Kiyavash, Negar"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2010,"date_issued":"2010-05-18T18:53:33Z","date_published":"2010-05-18T18:53:33Z","updated_at":"2026-07-22T22:25:08Z","subjects":["information theory","causality","computational neuroscience","Bayesian networks"],"languages":["en"],"rights":["Copyright 2010 Christopher John Quinn"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/15989","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Coleman, Todd P.","Kiyavash, Negar"]},{"key":"dc:creator","label":"Author","values":["Quinn, Christopher J."]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2010-05-18T18:53:33Z","2012-05-19T10:00:14Z","2010-5"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["information theory","causality","computational neuroscience","Bayesian networks"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2010 Christopher John Quinn"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/15989"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["\"This work examines an information theoretic quantity known as directed information, which measures statistically causal influences between processes. It is shown to be a general quantity, applicable to arbitrary probability distributions. It is interpreted in terms of prediction, communication with feedback, source coding with feed forward, control over noisy channels, and other settings. It is also shown to be consistent with Granger's philosophical definition. The concepts of direct and indirect causation in a network of processes are formalized. Next, two applications of directed information are investigated. Neuroscience researchers have been attempting to identify causal relationships between neural spike trains in electrode recordings, but have been doing so with correlation measures and measures based on Granger causality. We discuss why these methods are not robust, and do not have statistical guarantees. We use a point process GLM model and MDL (as a model order selection tool) for consistent estimation of directed information between neural spike trains. We have successfully applied this methodology to a network of simulated neurons and electrode array recordings. This work then develops a procedure, similar to Chow and Liu's, for fi nding the \"\"best\"\" approximation (in terms of KL divergence) of a full, joint distribution over a set of random processes, using a causal dependence tree distribution. Chow and Liu's procedure had been shown to be equivalent to maximizing a sum of mutual informations, and the procedure presented here is shown to be equivalent to maximizing a sum of directed informations. An algorithm is presented for efficiently finding the optimal causal tree, similar to that in Chow and Liu's work.\"","Item withdrawn by Mark Zulauf (zulauf@illinois.edu) on 2010-04-26T21:06:07Z Item was in collections: University of Illinois Theses & Dissertations (ID: 1) No. of bitstreams: 1 Quinn_Christopher.pdf: 746842 bytes, checksum: 215c52c35cdc2b7a0588d032d9a41e10 (MD5)","Made available in DSpace on 2010-05-18T18:53:33Z (GMT). No. of bitstreams: 2 Quinn_Christopher.pdf: 746842 bytes, checksum: 215c52c35cdc2b7a0588d032d9a41e10 (MD5) license.txt: 4065 bytes, checksum: eeb4f9f45cd5398608de15cf48b474cd (MD5)","Item marked as restricted to the 'UIUC Users [automated]' Group (id=2) by William Ingram (wingram2@illinois.edu) on 2010-05-18T18:54:51Z Item is restricted until 2012-05-18T18:54:47Z","Item reinstated by Sarah Shreeves (sshreeve@illinois.edu) on 2012-05-19T10:00:14Z Item was in collections: Dissertations and Theses - Electrical and Computer Engineering (ID: 446) University of Illinois Dissertations and Theses (ID: 204) No. of bitstreams: 3 Quinn_Christopher.pdf.txt: 121270 bytes, checksum: 306e2e96b2b7e68bcbfcae3c44ce1862 (MD5) Quinn_Christopher.pdf: 746842 bytes, checksum: 215c52c35cdc2b7a0588d032d9a41e10 (MD5) license.txt: 4065 bytes, checksum: eeb4f9f45cd5398608de15cf48b474cd (MD5)","Item released from any restrictions by Sarah Shreeves (sshreeve@illinois.edu) on 2012-05-19T10:00:14Z"]},{"key":"dc:title","label":"Title","values":["Estimating directed information to infer causal relationships between neural spike trains and approximating discrete probability distributions with causal dependence trees"]}]}],"canonical_facts":{"dc:contributor":["Coleman, Todd P.","Kiyavash, Negar"],"dc:creator":["Quinn, Christopher J."],"dc:date":["2010-05-18T18:53:33Z","2012-05-19T10:00:14Z","2010-5"],"dc:description":["\"This work examines an information theoretic quantity known as directed information, which measures statistically causal influences between processes. It is shown to be a general quantity, applicable to arbitrary probability distributions. It is interpreted in terms of prediction, communication with feedback, source coding with feed forward, control over noisy channels, and other settings. It is also shown to be consistent with Granger's philosophical definition. The concepts of direct and indirect causation in a network of processes are formalized. Next, two applications of directed information are investigated. Neuroscience researchers have been attempting to identify causal relationships between neural spike trains in electrode recordings, but have been doing so with correlation measures and measures based on Granger causality. We discuss why these methods are not robust, and do not have statistical guarantees. We use a point process GLM model and MDL (as a model order selection tool) for consistent estimation of directed information between neural spike trains. We have successfully applied this methodology to a network of simulated neurons and electrode array recordings. This work then develops a procedure, similar to Chow and Liu's, for fi nding the \"\"best\"\" approximation (in terms of KL divergence) of a full, joint distribution over a set of random processes, using a causal dependence tree distribution. Chow and Liu's procedure had been shown to be equivalent to maximizing a sum of mutual informations, and the procedure presented here is shown to be equivalent to maximizing a sum of directed informations. An algorithm is presented for efficiently finding the optimal causal tree, similar to that in Chow and Liu's work.\"","Item withdrawn by Mark Zulauf (zulauf@illinois.edu) on 2010-04-26T21:06:07Z Item was in collections: University of Illinois Theses & Dissertations (ID: 1) No. of bitstreams: 1 Quinn_Christopher.pdf: 746842 bytes, checksum: 215c52c35cdc2b7a0588d032d9a41e10 (MD5)","Made available in DSpace on 2010-05-18T18:53:33Z (GMT). No. of bitstreams: 2 Quinn_Christopher.pdf: 746842 bytes, checksum: 215c52c35cdc2b7a0588d032d9a41e10 (MD5) license.txt: 4065 bytes, checksum: eeb4f9f45cd5398608de15cf48b474cd (MD5)","Item marked as restricted to the 'UIUC Users [automated]' Group (id=2) by William Ingram (wingram2@illinois.edu) on 2010-05-18T18:54:51Z Item is restricted until 2012-05-18T18:54:47Z","Item reinstated by Sarah Shreeves (sshreeve@illinois.edu) on 2012-05-19T10:00:14Z Item was in collections: Dissertations and Theses - Electrical and Computer Engineering (ID: 446) University of Illinois Dissertations and Theses (ID: 204) No. of bitstreams: 3 Quinn_Christopher.pdf.txt: 121270 bytes, checksum: 306e2e96b2b7e68bcbfcae3c44ce1862 (MD5) Quinn_Christopher.pdf: 746842 bytes, checksum: 215c52c35cdc2b7a0588d032d9a41e10 (MD5) license.txt: 4065 bytes, checksum: eeb4f9f45cd5398608de15cf48b474cd (MD5)","Item released from any restrictions by Sarah Shreeves (sshreeve@illinois.edu) on 2012-05-19T10:00:14Z"],"dc:identifier":["http://hdl.handle.net/2142/15989"],"dc:language":["en"],"dc:rights":["Copyright 2010 Christopher John Quinn"],"dc:subject":["information theory","causality","computational neuroscience","Bayesian networks"],"dc:title":["Estimating directed information to infer causal relationships between neural spike trains and approximating discrete probability distributions with causal dependence trees"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:08Z"}