The Interaction Between Learning and Effort-Based Decision-Making: Mechanisms, Structural Brain Correlates, and Links to Neuropsychiatric Symptoms
We must learn about the inherent uncertainty in our environment to behave adaptively, and a key aspect of adaptive behaviour includes how we decide to allocate effort when making decisions. How we learn about our environment is therefore necessarily connected to how we make choices about effort. However, the literature on effort choice has largely neglected this connection. Studies have shown that across neuropsychiatric disorders, there are impairments in both learning and effort-based decision-making, though no studies to date have systematically examined whether the translation of learned expectations into effort might account for mood and anxiety disorder symptoms such as apathy and anhedonia. One brain region strongly implicated in both learning and motivated action is the habenula, which sits at the crossroads of the serotonergic learning circuits and dopaminergic control of behaviour. Habenula structural integrity has been implicated in both mood disorders and neurological conditions with co-occurring mood symptoms such as Parkinson’s disease (PD), but the relationship between habenula volume, learning and effortchoice is largely unknown. This thesis will investigate the association between expressions of learning and habenula volume, then present a novel experiment optimised to quantify the interaction between learning and effort decision-making. To investigate the relevance of neuropsychiatric symptoms to these behaviours, we collect data from the general population and a clinical sample before returning to the habenula volume and its relationship to learning and motivated behaviour. v Chapter 1 introduces the ideas that motivate the thesis, including transdiagnostic, computational/precision psychiatry approaches. Followed by a description of the relevant literature on learning, effort-based decision making and their neural basis. Then, the chapter will outline the transdiagnostic, neuropsychiatric relevance of these behaviours. In Chapter 2, the first empirical chapter, I re-analyse data from a reinforcement learning task and structural brain images from Parkinson’s Disease (PD) and matched controls to investigate the relationship between habenula volume and avoidance behaviour. The results demonstrate that PD patients have larger habenulas than controls. Those larger habenula volumes are related to greater avoidance behaviour, control analyses show this effect is specific to avoidance behaviour, not only present in the control group and unaffected by PD medication status. Chapter 3 describes the task development of the paradigm central to this thesis, which aims to characterise the interaction between learning and effort decision-making. Participants first learn the probabilistic associations between eight stimuli, with four associated with reward and four with loss and report their estimate of the reward probability on every trial. Then they complete a tournament task where all eight stimuli are paired together repeatedly to enable assessment of their learning. Finally, participants make accept/reject choices for a given learned stimulus and varying levels of effort. Participants can learn accurately about the outcome probability and magnitude of the stimuli then guide their effort decision-making using these variables in conjunction with effort level. Chapter 4 presents the data from the final version of the learning and effort choice task, collected in a large online sample (n=252). The larger dataset is leveraged to address questions about whether individual measures of learning better predict participants’ effort choice than the objective stimulus features and questions about the effect of transdiagnostic dimensions, anhedonia and fatigue on these behaviours. Individualised measures of learning outperform the objective outcome probabilities in the prediction of effort choice and anhedonia is associated with reduced effort vi exertion and an alteration in the integration of subjective beliefs about probability and effort choice. Chapter 5 aims to further understand the effects of neuropsychiatric symptoms on learning, effort decision-making and their interaction by recruiting a population of anxious/depressed participants and healthy controls to complete the task. Participants’ mental health symptoms are richly characterised then factor analysed. Individualised learning measures better predict participants’ effort choices than the objective outcome probabilities. Despite clear differences in symptoms between the clinical group and controls, they exhibit no clear differences in their learning and effort choice behaviour. In Chapter 6, we conducted an in-person replication of the learning and effort decision-making task to validate the findings from the earlier online studies (Chapters 3, 4 and 5). The behavioural effects observed online were replicated, including the main result of Chapter 4, as individual learning measures outperformed the objective outcome probabilities when predicting participants’ effort choices. In addition to the behavioural replication, we acquired 7T structural scans for each participant, to precisely delineate the habenula, offering a substantial improvement in spatial resolution compared to the 3T scans used in Chapter 2. These preliminary results allowed us to begin characterising how inter-individual variation in habenula structure relates to learning, effort decisions and their interaction. Finally, Chapter 7 situates these results in the wider literature and evaluates their contribution to the understanding of learning, effort decision-making, their interaction and neuropsychiatric relevance. Then, by reflecting critically on the empirical chapters, I posit some questions that future research might take up to advance the field.
Anhedonia, Anxiety, Apathy, Cognitive Neuroscience, Computational Psychiatry, Depression, Effort-based decision-making, Habenula, Learning, Mental Health, Motivation, Neuroscience, Parkinson's Disease, Psychology, Structural MRI, Uncertainty
Networked Digital Library of Theses and Dissertations