Universal approximation of input-output maps and dynamical systems by neural network architectures

It is well known that feedforward neural networks can approximate any continuous function supported on a finite-dimensional compact set to arbitrary accuracy. However, many engineering applications require modeling infinite-dimensional functions, such as sequence-to-sequence transformations or input-output characteristics of systems of differential equations. For discrete-time input-output maps having limited long-term memory, we prove universal approximation guarantees for temporal convolutional nets constructed using only a finite number of computation units which hold on an infinite-time horizon. We also provide quantitative estimates for the width and depth of the network sufficient to achieve any fixed error tolerance. Furthemore, we show that discrete-time input-output maps given by state-space realizations satisfying certain stability criteria admit such convolutional net approximations which are accurate on an infinite-time scale. For continuous-time input-output maps induced by dynamical systems that are stable in a similar sense, we prove that continuous-time recurrent neural nets are capable of reproducing the original trajectories to within arbitrarily small error tolerance over an infinite-time horizon. For a subset of these stable systems, we provide quantitative estimates on the number of neurons sufficient to guarantee the desired error bound.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2020

Author dc:creator

• Hanson, Joshua McKinley

Contributors dc:contributor

• Raginsky, Maxim

• Input-output maps
• convolutional neural nets
• dynamical systems
• recurrent neural nets
• deep neural networks
• continuous time
• discrete time
• universal approximation
• simulation
• feedback
• stability
• fading memory
• approximately finite memory

Statement dc:rights

• Copyright 2020 Joshua Hanson

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/108486

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/108486