Deep learning for prosody-based irony classification in spontaneous speech

Recognizing irony in speech and text can be challenging even for humans. For natural language processing applications, irony recognition presents a unique challenge as the presence of irony reverses the meaning and sentiment of the words themselves. Combining phonological insights and machine learning methods, this dissertation endeavors to capture as holistic as possible an acoustic impression of ironic speech as it differs from non-ironic speech, and use the insights gleaned to inform the construction of deep learning-based irony classification models. The Sad Boyz Corpus, consisting of 4.68 hours of irony-annotated, naturalistic, conversational speech data has been constructed for the purposes of this research, and is publicly available for future researchers in this space. A wide array of utterance-level and time-series acoustic features are extracted from this data, and are analyzed using PCA, Logistic Mixed Effect Regression Models, and Generalized Additive Models. Inferential statistical results reveal areas of statistically significant difference (p< 0.05) between ironic and non-ironic speech in the domains of frequency (F0) on both the utterance level and time-series contour, utterance-level timing/duration features, and time-series HNR, MFCCs, and RASTA-PLP. These inferential statistical results are then utilized to inform the training and fine-tuning of a series of deep learning approaches for irony classification, achieving an AUC of 0.8 in the speaker dependent condition, and an AUC of 0.77 in the speaker independent condition, outperforming the results of most irony classification models in the existing literature (Burkhardt et al., 2017; Castro et al., 2019). In addition to the myriad of real-world applications for such a system, its contribution to the understanding of prosodically-encoded augmentation of semantic content constitutes a significant step forward for research in the fields of linguistics and NLP.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Linguistics

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2022

Author dc:creator

• Gent, Helen

Contributors dc:contributor

• Shosted, Ryan
• Shih, Chilin
• Tang, Yan
• Girju, Corina R
• Schwartz, Lane

• irony classification
• prosody
• phonology
• naturalistic speech data
• deep learning
• prosody modeling
• speech data
• natural language processing
• speech processing
• naturalistic speech corpus
• computational paralinguistics
• computational linguistics
• linguistics
• pragmatics
• irony

Statement dc:rights

• Copyright 2022 Helen Gent

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/115362

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F115362