{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/108486"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/108486","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Universal approximation of input-output maps and dynamical systems by neural network architectures","abstract":"It is well known that feedforward neural networks can approximate any continuous function supported on a finite-dimensional compact set to arbitrary accuracy. However, many engineering applications require modeling infinite-dimensional functions, such as sequence-to-sequence transformations or input-output characteristics of systems of differential equations. For discrete-time input-output maps having limited long-term memory, we prove universal approximation guarantees for temporal convolutional nets constructed using only a finite number of computation units which hold on an infinite-time horizon. We also provide quantitative estimates for the width and depth of the network sufficient to achieve any fixed error tolerance. Furthemore, we show that discrete-time input-output maps given by state-space realizations satisfying certain stability criteria admit such convolutional net approximations which are accurate on an infinite-time scale. For continuous-time input-output maps induced by dynamical systems that are stable in a similar sense, we prove that continuous-time recurrent neural nets are capable of reproducing the original trajectories to within arbitrarily small error tolerance over an infinite-time horizon. For a subset of these stable systems, we provide quantitative estimates on the number of neurons sufficient to guarantee the desired error bound.","abstract_html":"It is well known that feedforward neural networks can approximate any continuous function supported on a finite-dimensional compact set to arbitrary accuracy. However, many engineering applications require modeling infinite-dimensional functions, such as sequence-to-sequence transformations or input-output characteristics of systems of differential equations. For discrete-time input-output maps having limited long-term memory, we prove universal approximation guarantees for temporal convolutional nets constructed using only a finite number of computation units which hold on an infinite-time horizon. We also provide quantitative estimates for the width and depth of the network sufficient to achieve any fixed error tolerance. Furthemore, we show that discrete-time input-output maps given by state-space realizations satisfying certain stability criteria admit such convolutional net approximations which are accurate on an infinite-time scale. For continuous-time input-output maps induced by dynamical systems that are stable in a similar sense, we prove that continuous-time recurrent neural nets are capable of reproducing the original trajectories to within arbitrarily small error tolerance over an infinite-time horizon. For a subset of these stable systems, we provide quantitative estimates on the number of neurons sufficient to guarantee the desired error bound.","abstract_has_math":false,"creators":["Hanson, Joshua McKinley"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Raginsky, Maxim"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2020,"date_issued":"2020-10-07T20:59:50Z","date_published":"2020-10-07T20:59:50Z","updated_at":"2026-07-22T22:24:48Z","subjects":["Input-output maps","convolutional neural nets","dynamical systems","recurrent neural nets","deep neural networks","continuous time","discrete time","universal approximation","simulation","feedback","stability","fading memory","approximately finite memory"],"languages":["en"],"rights":["Copyright 2020 Joshua Hanson"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/108486","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Raginsky, Maxim"]},{"key":"dc:creator","label":"Author","values":["Hanson, Joshua McKinley"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2020-10-07T20:59:50Z","2020-07-15","2020-08"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Input-output maps","convolutional neural nets","dynamical systems","recurrent neural nets","deep neural networks","continuous time","discrete time","universal approximation","simulation","feedback","stability","fading memory","approximately finite memory"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2020 Joshua Hanson"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/108486"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["It is well known that feedforward neural networks can approximate any continuous function supported on a finite-dimensional compact set to arbitrary accuracy. However, many engineering applications require modeling infinite-dimensional functions, such as sequence-to-sequence transformations or input-output characteristics of systems of differential equations. For discrete-time input-output maps having limited long-term memory, we prove universal approximation guarantees for temporal convolutional nets constructed using only a finite number of computation units which hold on an infinite-time horizon. We also provide quantitative estimates for the width and depth of the network sufficient to achieve any fixed error tolerance. Furthemore, we show that discrete-time input-output maps given by state-space realizations satisfying certain stability criteria admit such convolutional net approximations which are accurate on an infinite-time scale. For continuous-time input-output maps induced by dynamical systems that are stable in a similar sense, we prove that continuous-time recurrent neural nets are capable of reproducing the original trajectories to within arbitrarily small error tolerance over an infinite-time horizon. For a subset of these stable systems, we provide quantitative estimates on the number of neurons sufficient to guarantee the desired error bound.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2020-10-02 without embargo terms","The student, Joshua Hanson, accepted the attached license on 2020-07-13 at 17:55.","The student, Joshua Hanson, submitted this Thesis for approval on 2020-07-13 at 18:10.","This Thesis was approved for publication on 2020-07-15 at 09:26.","DSpace SAF Submission Ingestion Package generated from Vireo submission #15597 on 2020-10-02 at 15:13:16","Made available in DSpace on 2020-10-07T20:59:50Z (GMT). No. of bitstreams: 3 HANSON-THESIS-2020.pdf: 419080 bytes, checksum: 725f66d8cbb7b3bcc78d1d0731f00cbb (MD5) ecethesis.zip: 465344 bytes, checksum: 48ca50b57fa458a7685c5e947470f549 (MD5) LICENSE.txt: 4210 bytes, checksum: dfb0bce9eb4e52d1757ae4d0d53eb487 (MD5) Previous issue date: 2020-07-15"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Universal approximation of input-output maps and dynamical systems by neural network architectures"]}]}],"canonical_facts":{"dc:contributor":["Raginsky, Maxim"],"dc:creator":["Hanson, Joshua McKinley"],"dc:date":["2020-10-07T20:59:50Z","2020-07-15","2020-08"],"dc:description":["It is well known that feedforward neural networks can approximate any continuous function supported on a finite-dimensional compact set to arbitrary accuracy. However, many engineering applications require modeling infinite-dimensional functions, such as sequence-to-sequence transformations or input-output characteristics of systems of differential equations. For discrete-time input-output maps having limited long-term memory, we prove universal approximation guarantees for temporal convolutional nets constructed using only a finite number of computation units which hold on an infinite-time horizon. We also provide quantitative estimates for the width and depth of the network sufficient to achieve any fixed error tolerance. Furthemore, we show that discrete-time input-output maps given by state-space realizations satisfying certain stability criteria admit such convolutional net approximations which are accurate on an infinite-time scale. For continuous-time input-output maps induced by dynamical systems that are stable in a similar sense, we prove that continuous-time recurrent neural nets are capable of reproducing the original trajectories to within arbitrarily small error tolerance over an infinite-time horizon. For a subset of these stable systems, we provide quantitative estimates on the number of neurons sufficient to guarantee the desired error bound.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2020-10-02 without embargo terms","The student, Joshua Hanson, accepted the attached license on 2020-07-13 at 17:55.","The student, Joshua Hanson, submitted this Thesis for approval on 2020-07-13 at 18:10.","This Thesis was approved for publication on 2020-07-15 at 09:26.","DSpace SAF Submission Ingestion Package generated from Vireo submission #15597 on 2020-10-02 at 15:13:16","Made available in DSpace on 2020-10-07T20:59:50Z (GMT). No. of bitstreams: 3 HANSON-THESIS-2020.pdf: 419080 bytes, checksum: 725f66d8cbb7b3bcc78d1d0731f00cbb (MD5) ecethesis.zip: 465344 bytes, checksum: 48ca50b57fa458a7685c5e947470f549 (MD5) LICENSE.txt: 4210 bytes, checksum: dfb0bce9eb4e52d1757ae4d0d53eb487 (MD5) Previous issue date: 2020-07-15"],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/108486"],"dc:language":["en"],"dc:rights":["Copyright 2020 Joshua Hanson"],"dc:subject":["Input-output maps","convolutional neural nets","dynamical systems","recurrent neural nets","deep neural networks","continuous time","discrete time","universal approximation","simulation","feedback","stability","fading memory","approximately finite memory"],"dc:title":["Universal approximation of input-output maps and dynamical systems by neural network architectures"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:48Z"}