Dynamical systems perspectives in machine learning

We look at two facets of machine learning from a perspective of dynamical systems, that is, the data generated from a dynamical system and the iterative inference algorithm posed as a dynamical system. In the former, we look at time series data which is generated from a mixture of processes. Each process exists for a fixed duration and generates i.i.d categorical data points during that duration. More than one process can coexist at a particular time. The goal is to find the number of such hidden processes and the characteristic categorical distribution of each. This model is motivated by the problem of finding error events in error-logs from a mobile communication network. In the second direction, we consider the problem of regression using a shallow overparameterized neural network. Broadly, we look at training the neural network with the gradient descent algorithm on the squared loss function and discuss the generalization properties of the output of the gradient descent algorithm on an unseen data point. We look at two problems in this setting. First, we discuss the effect of l2 regularization on the squared loss and discuss how different strength of regularization provides a trade-off on the generalization of the neural network. Second, we look at squared loss without regularization and discuss the generalization properties when the true function we are trying to learn belongs to the class of polynomials in the presence of noisy samples. In both the problems, we consider the gradient descent algorithm as a dynamical system and use tools from control theory to analyze this dynamical system.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Satpathi, Siddhartha

Contributors dc:contributor

• Srikant, Rayadurgam
• Beck, Carolyn L
• Chatterjee, Sabyasachi
• Hu, Bin

• Neural Network
• error-log
• LDA
• gradient descent

Statement dc:rights

• Copyright 2021 Siddhartha Satpathi

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/113008

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/113008