Robust model-based reinforcement learning using L1 adaptive control

We introduce L1-MBRL, a control-theoretic augmentation scheme for Model-Based Reinforcement Learning (MBRL) algorithms. Unlike model-free approaches, MBRL algorithms learn a model of the transition function using data and use it to design a control input. Our approach approximates the transition function along each trajectory with a control-affine model to generate a control input augmentation that perturbs the input produced by the underlying MBRL policy. The perturbation produced by the L1 adaptive control is designed to enhance the robustness of the system against uncertainties. Importantly, the proposed L1 adaptive control-based learning scheme is agnostic to the choice of MBRL algorithm and can be integrated seamlessly with many model-based RL systems in practice. The method exhibits superior performance and sample efficiency on multiple MuJoCo environments, both with and without system noise, as demonstrated by our numerical simulations.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Aerospace Engineering

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2023

Author dc:creator

• Karumanchi, Sambhu Harimanas

Contributors dc:contributor

• Hovakimyan, Naira

• Robust Adaptive Control
• Reinforcement Learning

Statement dc:rights

• Copyright 2023 Sambhu Harimanas Karumanchi

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/121362

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F121362