Nonlinear adaptation of established linear and predictive control laws through safe reinforcement learning
type: CreativeWork
description: Reinforcement learning (RL) enables the prospect of data-driven controllers that learn to select control actions optimally purely through the feedback provided by an evaluative signal (the reward). In principle, this technology may be used to develop adaptive controllers that account for plant-model mismatches, thereby unlocking untapped potential in the operation of processes in the chemical- and minerals processing industries. RL relies on trial-and-error design of opaque function approximator architectures and the selection of hyperparameters. Furthermore, challenges in establishing RL agent training that is efficient and safe currently hinders the technology’s industrial adoption. The analysis of RL control is typically decoupled from well-established control principles derived from classical and optimal control research. The first objective of the study was to show that RL can be applied to the adaptive control of a visualizable control policy initialized using proportional-integral (PI) control. This included formulating an RL controller to accommodate continuous control without randomised starting states and incorporating safety into the model-free design. The second goal of the research was to relate the tuning of this controller to generalised plant dynamics. Thirdly, the strengths and weaknesses of RL were assessed by tasking the controller with the adaptation of explicit model predictive control (MPC) policies. A key strategy in achieving each objective was to provide a warm start to the RL agent’s policy by fitting a shallow neural network to a policy precomputed using linear control or non-linear MPC. Subsequently, RL was applied to regulatory controller adaptation. An early actor-critic algorithm foundational to modern RL was used throughout. Model-free RL was shown to enable adaptive control of a visualizable control policy initialised to emulate PI control. By incorporating a safety mechanism in the RL agent’s control policy, control performance impr…
abstract: Reinforcement learning (RL) enables the prospect of data-driven controllers that learn to select control actions optimally purely through the feedback provided by an evaluative signal (the reward). In principle, this technology may be used to develop adaptive controllers that account for plant-model mismatches, thereby unlocking untapped potential in the operation of processes in the chemical- and minerals processing industries. RL relies on trial-and-error design of opaque function approximator architectures and the selection of hyperparameters. Furthermore, challenges in establishing RL agent training that is efficient and safe currently hinders the technology’s industrial adoption. The analysis of RL control is typically decoupled from well-established control principles derived from classical and optimal control research. The first objective of the study was to show that RL can be applied to the adaptive control of a visualizable control policy initialized using proportional-integral (PI) control. This included formulating an RL controller to accommodate continuous control without randomised starting states and incorporating safety into the model-free design. The second goal of the research was to relate the tuning of this controller to generalised plant dynamics. Thirdly, the strengths and weaknesses of RL were assessed by tasking the controller with the adaptation of explicit model predictive control (MPC) policies. A key strategy in achieving each objective was to provide a warm start to the RL agent’s policy by fitting a shallow neural network to a policy precomputed using linear control or non-linear MPC. Subsequently, RL was applied to regulatory controller adaptation. An early actor-critic algorithm foundational to modern RL was used throughout. Model-free RL was shown to enable adaptive control of a visualizable control policy initialised to emulate PI control. By incorporating a safety mechanism in the RL agent’s control policy, control performance impr…
author: Bras, Edward Hendrik
language: en
date published: 2026-03
date modified: 2026-07-24T04:40:06Z
provider: Stellenbosch University
publisher: Networked Digital Library of Theses and Dissertations
type: WebPage
source organization: Stellenbosch : Stellenbosch University