A comparison of imaging modalities and decoding methods for detecting semantic information in the brain

Representation of semantic information enables us to engage with the world in a meaningful way – to comprehend and produce language, identify and use objects, and understand and participate in events that involve us. Our understanding of where in the brain semantic information is represented has progressed much more rapidly than our understanding of how semantic information is represented – that is, how activity in the brain enables semantic knowledge to be stored, ready for later deployment. This thesis aimed (1) to develop a theoretical framework with which to describe the nature of neural representations, including semantic representations; (2) to assess and compare the capacities of electrocorticography (ECoG) and 7 tesla functional magnetic resonance imaging (7T-fMRI) to detect semantic representations; and (3) to evaluate the strengths and limitations of multivariate analysis methods, in particular methods based on regularised regression, for revealing properties of semantic representations. In Chapter 2 I propose a new theoretical framework for describing representations. By posing six questions about the computational and neural characteristics of the representations that different theorists posit, I situate each contemporary theory relative to the others and bring the inseparable relationship between theory and analysis into focus – different multivariate methods encapsulate different assumptions (not always made explicit) about how the brain represents information. In Chapter 3 I investigate the temporal dynamics of semantic representations, specifically time-frequency power and phase. By analysing ECoG data recorded from grid electrodes on the ventral temporal cortical surface, I established that semantic information could be decoded from multiple frequency bands. However, only when classifiers were trained on power from all frequencies between 4 and 200 Hz did the “distributed, dynamic" properties observed in voltage data and in a computational model of semantic cognition emerge, suggesting that semantic information is represented in the ventrolateral anterior temporal lobe (vATL) in a “transfrequency” fashion. In Chapter 4 I lay the foundations for studying semantic representations with 7T-fMRI– I optimised a 7T-fMRI acquisition sequence that improved sensitivity in the vATL while maintaining sensitivity across the rest of the brain. I demonstrated that a multi-echo, multiband sequence achieves these aims. In Chapter 5 I used the acquisition sequence optimised in Chapter 4, plus four different multivariate decoding methods, to ask why semantic information is so rarely detected in the vATL with fMRI despite the body of evidence for its presence and to ask whether and where semantic information is represented elsewhere in the brain. Having found evidence for dynamic, graded, multidimensional representations in the vATL, I concluded that my use of a distortion-corrected acquisition sequence and my choice of analysis methods are the most likely reasons for the difference between my findings and previous work. I also found evidence of graded, multidimensional semantic structure in posterior temporal cortex. To conclude, this thesis (1) developed a unifying theoretical framework in which to situate theories about, and methods for discovering, semantic representations in the brain; (2) established that both ECoG and fMRI can provide insight into the properties of semantic representations; and (3) demonstrated that decoding methods that incorporate neurally-inspired regularisation penalties can be beneficial for decoding, but argued that the best decoding methods for future studies are those that are carefully selected to complement the research question.

Name dc:type.qualificationname

Doctor of Philosophy (PhD)

Level dc:type.qualificationlevel

Doctoral

Grantor dc:publisher.institution

University of Cambridge

Year dc:date.issued

2024

Author dc:creator

• Frisby, Saskia

Advisor dc:contributor.advisor

• Lambon Ralph, Matthew

• 7T-fMRI
• brain imaging
• cognitive neuroscience
• concepts
• decoding
• ECoG
• electrocorticography
• intracranial electrophysiology
• multiband
• multi-echo
• multivariate pattern analysis
• neural decoding
• parallel transmit
• semantic cognition
• semantic memory
• semantic representation
• time-frequency analysis

Licence

• https://www.repository.cam.ac.uk/bitstreams/18d3957c-3ea8-483e-8e88-232cfc1fa16f/download
• https://creativecommons.org/licenses/by/4.0/

Language dc:language

eng

Author Identifier

0000-0002-5786-6900

OAI identifier oai:identifier

oai:www.repository.cam.ac.uk:1810/379084