Integration of Control and Dynamical Systems Perspectives to Machine Learning

With the increasing demands on artificial intelligence technology operating over sequential data, represented by robotics and language processing, there has been a surge of interest in interdisciplinary research spanning machine learning -- a data-driven approach based on statistics -- and control or dynamical systems theory, which deals with dynamic environments. Because those streams of studies have evolved in a relatively separate manner under different settings and formulations, their integration becomes an intricate task, requiring a fresh look at the existing approach. This thesis primarily revolves around a discussion of research endeavors in the intersection of machine learning and dynamical systems to exploit the best of both worlds, and proposes some of the novel techniques and paradigms made possible by bringing the unique perspectives and concepts from these domains creatively. I initially provide a succinct overview of the state of the art in the related domains followed by my contributions to the fields. First of all, this thesis begins with the work that synthesizes control tools in a learning system to devise algorithms with control theoretic guarantees. In this process, a novel control concept, limited-duration safety, is proposed with discussions on its application within the context of transfer learning. Secondly, a novel model-based reinforcement learning (RL) algorithm is presented, leveraging a recent control theoretic tool as an oracle embedded in the algorithm to provably ensure learning efficiency. With a novel problem formulation with the Koopman operator, which is cast as a generalization of pole assignments to nonlinear decision making, a diverse array of dynamic behaviors are realized. Thirdly, as an additional highlight of exploration, I present the successful extension of RL beyond its conventional reliance on the Bellman equation, encompassing dynamic programming across entire paths. The new framework grounded in theoretical advancements of path signatures has proven beneficial in addressing challenges related to path following. On the other hand, merging machine learning, rooted in statistics, and dynamical systems raises several challenges. In particular, fourthly, this thesis discusses a specific challenge of loss of dynamic structure information that might be caused by concentrations of measures, which is overcome by carefully adopting the asymptotic results of exponential sums. Lastly, a machine learning algorithm itself can be seen as a dynamical system, and this perspective has theoretical and practical potential for handling complex machine learning domains. Especially for a deep RL algorithm, the constructive approach is taken to analyze, in a retroductive manner, the phenomena and performance separations observed in the systems of interest. This thesis is also intended to open a novel direction of further research emerging out of amalgamations of learning algorithms and dynamical systems perspectives, and is concluded with a remark for the potential future work.

Author dc:creator

• Ohnishi, Motoya

Advisors dc:contributor.advisor

• Kakade, Sham
• Todorov, Emanuel

• Control
• Dynamical systems
• Machine learning
• Robotics
• Computer science
• Electrical engineering
• Applied mathematics

Statement dc:rights

• none

Language dc:language.iso

en_US

Handle dc:identifier.uri

https://hdl.handle.net/1773/51882

OAI identifier oai:identifier

oai:digital.lib.washington.edu:1773/51882