Advancing System Models of Brain Processing via Integrative Benchmarking

Research in the brain and cognitive sciences attempts to uncover the neural mechanisms underlying intelligent behavior in domains such as vision or language. Due to the complexities of brain processing, studies necessarily had to start with a narrow scope of experimental investigation and computational modeling. This thesis argues that it is time for our field to take the next step: build system models that capture neural mechanisms and supported behaviors within an entire domain of intelligence. To make progress on system models, we propose integrative benchmarking – integrating experimental results from many laboratories into suites of benchmarks that guide and constrain those models at multiple stages and scales. We show-case this approach by developing Brain-Score benchmark suites for neural and behavioral experiments in the primate visual ventral stream and the human language system, as well as direct neural (causal) perturbation experiments in inferotemporal cortex. By systematically evaluating a wide variety of model candidates, we not only identify models beginning to match a range of brain data (∼ 50% explained variance), but also discover key relationships: Models’ brain scores are predicted by their object categorization performance in vision (but only up to 70% ImageNet accuracy), and their next-word prediction performance in language. The better models predict internal neural activity, the better they match human behavioral outputs, with architecture substantially contributing to brain-like representations. Using the integrative benchmarks, we develop improved state-of-the-art system models that more closely match shallow recurrent neuroanatomy and predict primate temporal processing, as well as models that require only a fraction of supervised synaptic updates. Taken together, the integrative benchmarks and system models presented here are first steps to modeling the complexities of brain processing in entire domains of intelligence.

Name thesis:degree_name

Doctoral

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2022

Author dc:creator

• Schrimpf, Martin

Advisor dc:contributor.advisor

• DiCarlo, James J.

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright MIT

Licence dc:rights.uri

• http://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/145597

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/145597