The integration of facial and auditory affect: An emotional McGurk effect?
type: CreativeWork
description: Research investigating human emotion processing has typically studied either auditory (vocal) or visual (facial) information in isolation. However, speech perception literature supports integration of auditory and visual phonetic information by way of the McGurk effect. This study hypothesized an emotional McGurk effect (i.e., integration of auditory and visual emotional information). To evaluate this hypothesis, emotionally incongruent auditory-visual stimuli (e.g., a joyful voice with a sad face) were developed on a sample of 120 participants. These stimuli were then presented to 30 additional participants who categorized them according to one of eight common emotions. Results indicated significant (p < .001) Condition by Emotion interaction effects, suggesting that emotional information from auditory and visual sources is integrated during the perception of auditory emotion. This integration appears to occur in a predictable manner. To understand emotion perception, future studies will need to consider the interaction of auditory and visual emotional information.
abstract: Research investigating human emotion processing has typically studied either auditory (vocal) or visual (facial) information in isolation. However, speech perception literature supports integration of auditory and visual phonetic information by way of the McGurk effect. This study hypothesized an emotional McGurk effect (i.e., integration of auditory and visual emotional information). To evaluate this hypothesis, emotionally incongruent auditory-visual stimuli (e.g., a joyful voice with a sad face) were developed on a sample of 120 participants. These stimuli were then presented to 30 additional participants who categorized them according to one of eight common emotions. Results indicated significant (p < .001) Condition by Emotion interaction effects, suggesting that emotional information from auditory and visual sources is integrated during the perception of auditory emotion. This integration appears to occur in a predictable manner. To understand emotion perception, future studies will need to consider the interaction of auditory and visual emotional information.
author: Forrest, Teri Jean
language: English
date published: 2000-01-01T08:00:00Z
date modified: 2026-07-24T05:25:18Z
provider: University of Nevada - Las Vegas
publisher: Networked Digital Library of Theses and Dissertations
type: WebPage
source organization: University of Nevada, Las Vegas

PropertyValue
property ID: DOI
value: 10.25669/we75-4r24