Interactions Between Syntax and Semantics in Language Models

How do syntax and semantics interact in neural language models? Is the intuitive dichotomy between semantics and syntax useful as a mental model of their behavior? In this thesis, I systematically investigate how models handle the interactions between sentence corruptions of each kind. I develop a setup for generating “syntactic” and “semantic” corruptions and combining them. Using this setup, I develop a general framework for understanding how classes of corruptions interact with each other, arguing that this can be understood in terms of a single “entanglement” parameter. This allows me to quantify entanglement between syntax and semantics in a variety of situations. The result is that semantics is more entangled with syntax than it is with itself, and that syntax is highly self-entangled.

Name thesis:degree_name

Master

Department dc:contributor.department

Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2022

Author dc:creator

• Bau, Anthony

Advisor dc:contributor.advisor

• Andreas, Jacob

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright MIT

Licence dc:rights.uri

• http://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/144680

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/144680