Selective attention for audiovisual integration of speech

The perceptual brain decomposes the audiovisual world into a set of audiovisual features such as color, shape, and pitch. An important question in perception science is whether selective attention is required to bind audiovisual features back into unified perceptual objects. In visual displays, targets defined as conjunctions of bound features typically cannot be searched for in parallel across a complex scene. Instead, attention must be scanned through a visual scene to find such targets. This means that conjunction of features requires selective attention. Only a few prior studies have investigated this process in the crossmodal audiovisual case. These prior studies left some ambiguity as to how temporal dynamics interact with selective attention, and none have used dynamic audiovisual speech as stimuli. In two experiments, this thesis explored whether the crossmodal conjunction of audio and visual speech features requires selective attention. In Chapter 2, we presented observers with displays of multiple visual faces and one audible voice. The task was to determine whether one of the faces was saying a sentence that matched the voice. In Chapter 3, we presented observers with displays of multiple audible voices and one visual face. Similarly, the task was to determine whether one of the voices was saying the sentence that matched the lip movements of the visual face. We compared response times to complete the task as the number of distracting items was increased. We found that it took longer to complete tasks with increasing number of distractors. This means that audiovisual speech targets cannot be registered in parallel across the scene and strongly suggests that selective attention is required to bind speech features across modalities.

Authors

• Boutros, Sylvain John
• University of Lethbridge. Faculty of Arts and Science

• neuroscience
• audiovisual
• psychology
• cognitive science
• auditory
• visual
• visual perception
• auditory perception
• speech integration
• Neurosciences
• Cognitive neuroscience
• Speech perception
• Selectivity (Psychology)
• Human information processing
• Attention
• Distraction (Psychology)
• Dissertations
• Academic

Identifier

hdl:10133/6445

OAI identifier oai:identifier

oai:opus.uleth.ca:10133/6445