Learning and adaptation in graphs, networks, and autonomous systems

Reinforcement learning and adaptation are widely used in a variety of applications, whenever we want to control a dynamical system in some optimal manner. For example, in wireless networking, we can use these tools to learn to route packets more efficiently, leading to lower latency. In games, we can learn better strategies through self-play. In this dissertation, we focus on three problems of reinforcement learning and adaptation. We first consider a theoretical problem in reinforcement learning called optimistic policy iteration (OPI). We prove convergence of a variant of OPI whose convergence properties have been previously unknown. Next, we consider the problem of designing an algorithm to allow a car to autonomously merge onto a highway from an on-ramp. Two broad classes of techniques have been proposed to solve motion planning problems in autonomous driving: Model Predictive Control (MPC) and Reinforcement Learning (RL). In this dissertation, we present an algorithm which blends the model-free RL agent with the MPC solution and show that it provides better trade-offs between a number of relevant metrics: passenger comfort, efficiency, crash rate and robustness. Finally, we analyze a wireless scheduling problem with a limited probing constraint. We show how a throughput optimal solution can be computed, even without knowledge of the channel statistics. Interestingly, although this problem requires adaptability in the face of unknown conditions, we find that reinforcement learning is not needed for optimal performance and may even be harmful if applied without care.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Lubars, Joseph

Contributors dc:contributor

• Srikant, Rayadurgam
• Beck, Carolyn L
• Hu, Bin
• Varshney, Lav R

• Engineering

Statement dc:rights

• Copyright 2021 Joseph Lubars

Language dc:language

en, eng

Handle dc:identifier

http://hdl.handle.net/2142/113817

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F113817