Adaptive intelligence: Integrating recurrence, deep reinforcement learning, and neuro-adaptive techniques for dynamic unified guidance & control of adaptive systems

In modern control systems, the search for algorithms that provide adaptive, unified guidance and control (G&C) capabilities remains a significant challenge. Traditional control mechanisms, such as LQR or PID, often exhibit limitations in performance, especially when compared to cutting edge reinforcement learning algorithms in simulations. This thesis delves deep into the potential of Deep Reinforcement Learning, specifically the Proximal Policy Optimization (PPO) algorithm, aiming to provide a unified G&C solution that surpasses conventional methods in both adaptability and performance. Our research embarked on an ambitious goal: to engineer a guidance and control algorithm with inherent adaptive capabilities. Central to this endeavor was the integration of a Gated Recurrent Unit (GRU) system identification network with PPO. This synthesis allowed us to concatenate the output from the GRU system identification with the state, subsequently feeding the enriched information into both actor and critic networks. As applied to the Lunar Lander Continuous environment, this novel setup enabled optimal flight performance even in the face of unexpected thruster failures. Comparative analysis revealed that the PPO integrated with GRU system identification (sysID) notably enhanced the performance, with the critic network loss converging to zero - a feat not achieved with PPO alone. Moreover, PPO with the addition of GRU system identification demonstrated a markedly higher success rate in lunar landings when juxtaposed against its PPO-only counterpart. Further advancements in adaptive control were explored through the Recurrent Deep Model Reference Adaptive Control (R-DMRAC) algorithm. Our results in the simulated hexcopter environment illuminated that R-DMRAC, with its recurrent neural network foundation, consistently outperformed both DMRAC and traditional MRAC, especially in scenarios characterized by highly nonlinear, state, and time dependent disturbances. The broader implications of our study underscore the transformative potential of incorporating recurrent neural networks into adaptive algorithms. Whether directly deployed as actor and critic networks or used in tandem with other architectures, as demonstrated in our PPO with the addition of GRU system identification and R-DMRAC experiments, recurrent elements markedly boost performance in the adaptive domain. This revelation paves the way for future research and real-world applications where adaptiveness and resilience to dynamic disturbances are paramount.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Aerospace Engineering

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2023

Author dc:creator

• Bout, Scott

Contributors dc:contributor

• Chowdhary, Girish
• Williams, Kyle
• Schlossman, Rachel

• Deep Learning
• Deep Reinforcement Learning
• Neuro-adaptive Control
• Recurrent Neural Networks
• Deep Neural Networks
• Machine Learning
• Guidance
• Control
• Adaptive Control

Statement dc:rights

• Copyright 2023 Scott Bout

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/122073

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F122073