Contributions of Reward Identity and Time Prediction Errors to Pavlovian Learning
type: CreativeWork
description: Some models of associative learning attempt to explain effects such as blocking and unblocking as reflecting a prediction error (PE): Associative strength, and thereby learning, are said to be greater when the unconditioned stimulus (US) received following a cue deviates from one’s expectations. Some models of PE-motivated behavior and PE-relevant brain activity represent the US as a single quantitative variable for outcome value, and that learning therefore only occurs when outcome value differs from expectations. There is growing evidence to suggest that changes in other dimensions of reward, such as identity and timing, also contribute to learning. In Experiment 1, we ran a blocking/unblocking study where both of these reward domains were unexpectedly reversed upon the introduction of novel stimuli during compound training, without affecting reward value. We then probed the nature of this learning using outcome-specific Pavlovian-to-Instrumental (PIT) testing. PEs of both reward identity and timing enabled learning towards the novel stimuli that would not have otherwise occurred (an “unblocking” effect), and this learning was specific to each reward. In Experiment 2, we trained rats on two stimulus-reward pairs before unexpectedly switching the identities of the rewards that followed each stimulus. Immediately after this reversal, brain tissue was fixed and stained for activity-dependent protein phosphorylation (p-rPS6) and the dopamine (DA) precursor tyrosine hydroxylase (TH). Reward identity PEs did not result in activation of neurons in the orbitofrontal cortex (OFC), basolateral amygdala (BLA), or any component of the striatum. In contrast, reward identity PEs did result in the recruitment of larger neural ensembles in a specific area of the ventral tegmental area (VTA). This effect was not specific to DA neurons. The implications of these findings for PE learning models and the brain structures involved are discussed.
abstract: Some models of associative learning attempt to explain effects such as blocking and unblocking as reflecting a prediction error (PE): Associative strength, and thereby learning, are said to be greater when the unconditioned stimulus (US) received following a cue deviates from one’s expectations. Some models of PE-motivated behavior and PE-relevant brain activity represent the US as a single quantitative variable for outcome value, and that learning therefore only occurs when outcome value differs from expectations. There is growing evidence to suggest that changes in other dimensions of reward, such as identity and timing, also contribute to learning. In Experiment 1, we ran a blocking/unblocking study where both of these reward domains were unexpectedly reversed upon the introduction of novel stimuli during compound training, without affecting reward value. We then probed the nature of this learning using outcome-specific Pavlovian-to-Instrumental (PIT) testing. PEs of both reward identity and timing enabled learning towards the novel stimuli that would not have otherwise occurred (an “unblocking” effect), and this learning was specific to each reward. In Experiment 2, we trained rats on two stimulus-reward pairs before unexpectedly switching the identities of the rewards that followed each stimulus. Immediately after this reversal, brain tissue was fixed and stained for activity-dependent protein phosphorylation (p-rPS6) and the dopamine (DA) precursor tyrosine hydroxylase (TH). Reward identity PEs did not result in activation of neurons in the orbitofrontal cortex (OFC), basolateral amygdala (BLA), or any component of the striatum. In contrast, reward identity PEs did result in the recruitment of larger neural ensembles in a specific area of the ventral tegmental area (VTA). This effect was not specific to DA neurons. The implications of these findings for PE learning models and the brain structures involved are discussed.
author: Siegel, Daniel B.
keywords: Behavioral Neurobiology, Experimental Analysis of Behavior, learning, prediction error, pavlovian, blocking, unblocking, dopamine
date published: 2021-09-01T07:00:00Z
date modified: 2026-07-24T02:00:35Z
provider: City University of New York - Graduate Center
publisher: Networked Digital Library of Theses and Dissertations
type: WebPage
source organization: The Graduate School and University Center of The City University of New York