Semantic-prosodic correlates of backchannel utterances in human-computer dialog

Backchannels, words like “uh-huh” and “yeah” in English, serve to provide feedback from listeners to speakers without stopping the speaker from continuing their current utterance or holding the floor. While the function of backchannels to communicate agreement, belief, concern, interest, understanding, and acknowledgement has been identified in former research, there has been little research into how the manner of their articulation translates into these perceived meanings. This dissertation examines these connections between prosody and semantics. An audio corpus of the backchannel utterances “uh-huh” and “yeah” has been constructed to support this research. The corpus features voice actors purposefully inflecting backchannels across a range of semantic dimensions. Acoustic features of pitch, duration, energy, and the pitch energy synchrony are extracted from these samples. A set of predictive models based on multiple linear regression is developed. Significant features are then used to build a minimalist synthesis model. Synthetic backchannels are created using this synthesis model and are then evaluated by other human subjects. In addition to synthesis, a proposed prosodic sub-language is investigated, and several rules of backchannel prosody are proposed that may align to and be grounded in existing research in the field of embodied cognition. The research identifies key features of backchannel prosody that act as relatively strong semantic correlates. And, that these featural guideposts can be modeled and modulated in such a way as to reliably encode semantic intent into the sample backchannels through prosody alone.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Informatics

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2023

Author dc:creator

• Chandler, Ryan Lee

Contributors dc:contributor

• Shih, Chilin
• Hasegawa-Johnson, Mark
• Girju, Roxana
• Dell, Gary
• Hummel, John

• Prosody
• Backchannels
• Back Channels

Statement dc:rights

• Copyright 2023 Ryan Lee Chandler

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/120270

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F120270