Trends, problems, and solutions in causality and reinforcement learning

This thesis reviews, examines, and investigates the trends in the fields of causality and in reinforcement learning (RL). Theory is developed for both active research areas, with a specific focus on the overlap in underlying theory. The core argument is that the RL problem can be formulated as a causal problem, where the agent is learning causal policies that maximise return (via some causal relationship implied by the policy) and does this via selecting optimal actions (performing interventions) in the environment. Although relevant in both model-based and model-free scenarios, focus is placed on model-based modalities where one can view the various models as being causal models. It is further argued that this reformulation enables various theoretical improvements in reasoning ability for a learning agent, and does this while offering improved efficiency, interpretability, robustness, and generalisation across various learning modalities. As an application of the causal methods discussed, we also investigate whether applied causal discovery can lead to disparate impacts on sensitive subgroups. Finally, we reflect on the findings, highlight open problems, and propose future research directions.

Grantor dc:publisher.institution

Department of Mathematics and Applied Mathematics

Year dc:date.issued

2024

Author dc:creator

• Grimbly, St John

Advisor dc:contributor.advisor

• Shock, Jonathan

• Mathematics and Applied Mathematics

Handle dc:identifier.uri

http://hdl.handle.net/11427/40948

OAI identifier oai:identifier

oai:open.uct.ac.za:11427/40948