When the Brain Plays a Game: Neural Responses to Visual Dynamics during Naturalistic Visual Tasks

<p>Many day-to-day tasks involve processing of complex visual information in a continuous stream. While much of our knowledge on visual processing has been established from reductionist approaches in lab-controlled settings, very little is known about the processing of complex dynamic stimuli experienced in everyday scenarios. Traditional investigations employ event-related paradigms that involve presentation of simple stimuli at select locations in visual space and discrete moments in time. In contrast, visual stimuli in real-life are highly dynamic, spatially-heterogeneous, and semantically rich. Moreover, traditional experiments impose unnatural task constraints (e.g., inhibited saccades), thus, it is unclear whether theories developed under the reductionist approach apply in naturalistic settings. Given these limitations, alternative experimental paradigms and analysis methods are necessary. Here, we introduce a new approach for investigating visual processing, applying the system identification (SI) framework. We investigate the modulation of stimulus-evoked responses during a naturalistic task (i.e., kart race game) using non-invasive scalp recordings.</p> <p>In recent years, multivariate modeling approaches have become increasingly popular for assessing neural response to naturalistic stimuli. Encoding models use stimulus patterns to predict brain responses and decoding models use patterns of brain responses to predict stimulus that drove these responses. In this dissertation, we employ a hybrid method that “encodes” the stimulus to predict “decoded” brain responses. Using this approach, we measure the stimulus-response correlation (SRC), i.e. temporal correlation of neural response and dynamic stimulus. This SRC can be used to assess the strength of stimulus-evoked activity to uniquely experienced naturalistic stimulus. To demonstrate this, we measured the SRC during a kart race videogame. We find that SRC increased with active play of the game, suggesting that stimulus-evoked activity is modulated by the visual task demands. Furthermore, we analyzed the selectivity of neural response across the visual space. While it is well-established that neural response is spatially selective to discrete stimulus, it is unclear whether this is true during naturalistic stimulus presentation. To assess this, we measured the correlation of neural response with optical flow magnitude at individual locations on the screen during the videogame. We find that the SRC is greater for locations in space that are task-relevant, enhancing during active play. Moreover, the spatial selectivity differs across scalp locations, which suggest that individual brain regions are spatially selective to different visual dynamics.</p> <p>In summary, we leverage the SI framework to investigate visual processing during a naturalistic stimulus presentation, extending visual research to ecologically valid paradigms. Moreover, we demonstrate spatial selectivity of neural response that are task-relevant. Overall, our findings shed new insights about the stimulus-evoked neural response to visual dynamics during a uniquely experienced naturalistic visual task. Taken together, this dissertation work makes a significant contribution towards understanding how visual dynamics and task behavior affects neural responses in naturalistic conditions.</p>

Name thesis:degree_name

Doctor of Philosophy (Ph.D.)

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Biomedical Engineering

Year dc:date.available

2021

Author dc:creator

• Ki, Jason

Contributors dc:contributor

• Lucas C Parra
• Jacek Dmochowski

• Neuroscience
• Naturalistic Vision
• Visual Attention
• EEG
• Bioelectrical and Neuroengineering
• Bioimaging and Biomedical Optics
• Cognitive Neuroscience
• Computational Neuroscience
• Vision Science

Repository record dc:identifier

https://academicworks.cuny.edu/cc_etds_theses/930

OAI identifier oai:identifier

oai:academicworks.cuny.edu:cc_etds_theses-1966