A Neurophysiological Investigation of Listening Effort in Normal Hearing Adults Using fNIRS and Pupillometry

<p>Real-world conversations are often accompanied by some sort of interference that challenges the clarity of the speaker’s message, causing listeners to exert more effort to understand speech. Previous research has demonstrated that when listening to speech becomes difficult, various regions of the brain are recruited beyond those which engage during optimal listening conditions. However, the neural correlates that underly listening effort are not fully understood. Importantly, the pupillary response can be used to index listening effort, such that pupil size increases with increasing cognitive demand. I proposed that pupillometry can be used to characterize the cortical response, such that changes in pupil size would be associated with neural activation that directly relates to effortful processing. The primary goal of this study was to investigate the physiological mechanisms underlying recognition of realistic cochlear implant (CI) speech simulations (i.e., vocoded speech) and examine how effort might be alleviated with access to semantic information. To achieve this, I implemented a speech recognition task and manipulated the semantic content of the sentence (Predictability), spectral degradation (Speech Quality), and the way participants reported their response (Task Mode). Concurrent measurements of speech recognition, pupillary responses, and cortical activation via functional near infrared spectroscopy (fNIRS) were recorded from 41 normal hearing (NH) adults. As expected, challenging vocoded speech resulted in larger pupil sizes than did non-vocoded speech, and use of semantic information reduced listening effort. Interestingly, the largest exertion of listening effort was produced after the sentence presentation ended. Neural data revealed Predictability effects in frontal, temporal, and parietal cortices, such that activation in frontal regions responded to higher-level semantic representations of speech, and posterior brain regions were more sensitive to Task Mode which modulated the cortical mechanisms used to resolve Predictability. Positive correlations between the strength of the neural response and peak pupil response were observed in the Middle Frontal Gyrus (MFG) and Inferior Parietal Cortex (IPC), revealing cortical regions related to listening effort. Surprisingly, pupil size just prior to the sentence onset positively predicted individual speech recognition score, suggesting that the degree to which participants were engaged (or prepared) at the start of the trial predicted their overall speech recognition performance. On the other hand, pupil size right after sentence offset was negatively associated individual capacity to exploit semantic information, suggesting that listeners with smaller improvement from Low- to High-Predictability conditions had engaged more effort after the sentence was presented. Overall, the findings presented here indicate that the MFG and IPC carry out cortical mechanisms related to effortful processing, and listeners with a stronger MFG response were more impacted by the degraded listening conditions as they exhibited more effort (larger pupil size) and lower benefit in performance from semantic information. Therefore, it appears that the effect of listening effort on communication (i.e., speech recognition score and semantic capacity) is determined by the individual physiological response of the listener.</p>

Name thesis:degree_name

Doctor of Philosophy (PhD)

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Speech and Hearing Science

Year dc:date.available

2022

Author dc:creator

• Defenderfer, Jessica

Contributors dc:contributor

• Mark Hedrick, PhD

• fNIRS
• listening effort
• neuroimaging
• Pupillometry
• semantic information
• Speech Recognition
• Analytical, Diagnostic and Therapeutic Techniques and Equipment
• Investigative Techniques
• Medical Neurobiology
• Medical Physiology
• Medicine and Health Sciences
• Musculoskeletal, Neural, and Ocular Physiology
• Neurosciences
• Physiological Processes
• Speech and Hearing Science
• Speech Pathology and Audiology

Repository record dc:identifier

https://dc.uthsc.edu/dissertations/585

OAI identifier oai:identifier

oai:dc.uthsc.edu:dissertations-1590