Towards the processing of semantic idiomaticity in natural language

Pre-trained language models (PTLMs) are the cornerstone of recent breakthroughs in natural language processing (NLP). They enable machines to comprehend, analyze, and utilize human language in a multitude of applications. Despite their impressive abilities, PTLMs face challenges when dealing with idiomatic expressions (IEs), a common form of figurative language. IEs exhibit semantic non-compositionality, meaning that their figurative meaning cannot be derived from their constituent words, and contextual ambiguity, meaning that they can be interpreted literally depending on the context. This dissertation aims to explore the challenges posed by IEs to PTLMs and to advance the development of more effective NLP systems capable of processing semantic idiomaticity both explicitly and implicitly. To address the problem explicitly, we propose methods to detect and remove IEs from natural sentences and thus directly reduce the influences from IEs. Specifically, we investigate an IE identification approach that detects and localizes expressions that exhibit non-compositionality and an IE paraphrasing method that replaces the expressions with their literal counterpart phrases. To address the issue implicitly, we tackle the more fundamental issue of IE comprehension by enabling effective IE representation and introducing commonsense knowledge to PTLMs. We study methods to enhance PTLMs' ability to produce semantically meaningful and contextually appropriate embeddings for IEs. Additionally, we create an IE-centered commonsense knowledge graph and train PTLMs to become commonsense knowledge models capable of understanding and generating inferential knowledge on IE uses. Our research shows that with enhanced innate IE comprehension ability, PTLMs' performance on IE processing tasks and IE-related language understanding tasks would improve naturally. By pursuing both explicit and implicit approaches to processing semantic idiomaticity, our research has accomplished the overarching goal of extending the foundational capacity of PTLMs to perform natural language understanding in the presence of IEs.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2023

Author dc:creator

• Zeng, Ziheng

Contributors dc:contributor

• Bhat, Suma
• Varshney, Lav
• Hasegawa-Johnson, Mark
• Viswanath, Pramod

• Natural Language Processing
• Idiomatic Expression
• Non-compositional Language
• Idiom Identification
• Paraphrase Generation
• Representation Learning
• Commonsense Knowledge Graph

Statement dc:rights

• Copyright 2023 Ziheng Zeng

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/120501

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F120501