Distributionally robust L1 adaptive control for nonlinear ITˆO diffusion processes

The problem of optimal control has been studied starting from the 17th century. Since then, people have developed different approaches for solving the optimal control problem and applied the theory of optimal control in many areas, including economics, engineering, and operations research. However, those approaches assume people have full knowledge regarding the dynamical system for the optimal control problem, which is not always possible due to the randomness and uncertainties in the real world. In recent years, data-driven approaches have been established to solve the optimal control problem with partially known dynamical systems. One of the approaches is known as reinforcement learning, which is a branch of machine learning using rewards for desired or undesired behavior. Model-based reinforcement learning, as one way for the agent to learn optimal behaviors, is more widely accepted because of higher data efficiency. To capture more information about the model during the learning process for better performance, it is also suggested the agent uses learning from distributions rather than point estimation. However, due to the uncertainties, the learned model is likely to have a distribution that is different from the true model, which can cause the agent to perform poorly in the real world and may lead to dangerous consequences. The thesis considers the problem of errors in distributions between the learned model and the true model during the learning process from the control perspective and presents an approach to measure the difference between distributions as well as to provide a bound for the difference that can guarantee the performance for the agent. This thesis uses a continuous-time nonlinear stochastic system driven by the Wiener process. The system has the initial condition sampled from a distribution, and also has uncertainties in both the drift function part and the diffusion function part. The L1 adaptive controller is introduced for such a class of systems. Inside the L1 system, another Wiener process is introduced into both the reference system and the ideal system. Both systems have the same initial condition and corresponding initial condition distribution. The performance of L1 adaptive controller is then analyzed. A mean-square distance bound is provided between trajectories in actual and reference systems as well as trajectories in reference and ideal systems based on incremental Lyapunov functions. Furthermore, a bound between distributions behind trajectories between actual and ideal systems is subsequently provided. Simulation results are demonstrated how the controller can be used with traditional motion planning algorithms for obtaining safe trajectories. The code for the simulation is available at: https://github.com/SitaoZhang/StochasticMotionPlanning.jl

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Mechanical Engineering

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Zhang, Sitao

Contributors dc:contributor

• Hovakimyan, Naira

• Stochastic systems
• Adaptive control

Statement dc:rights

• Copyright 2022 Sitao Zhang

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/116109

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F116109