Computational Models for Binaural Sound Source Localization and Sound Understanding

As one of humans' primary sensors, the auditory system plays an important role in language acquisition. Computational models for binaural sound source localization and sound source understanding are proposed in this thesis. The models build a fundamental auditory system for a mobile robot that will automatically learn language through multisensory inputs and interaction with the external environment. A hypothesis-driven approach is followed for the localization model. Using only binaural inputs, it enables three-dimensional (3D) localization by combining multiple cues. Two binaural localization cues, interaural time differences (ITDs) and interaural intensity differences (IIDs), and one monoaural localization cue, spectral cues, are extracted from the input sounds. A Bayes rule-based hierarchical framework is applied for decision making. Simulations show the effectiveness of the model. A robust ITD estimation algorithm is introduced and implemented on the robot. Satisfactory results are achieved under real-world environments. A multimodal learning scheme is proposed with the aid of vision to realize autonomous learning for the 3D binaural localization. No human instructors need to be involved. A generic model is presented for sound source understanding. No labelled training data is required to build the model. A histogram is employed as the sound representation, where the time-varying characteristics of sound can be preserved. Histogram intersection is used as the similarity measurement between different sounds. The model is successfully applied to content-based audio information retrieval and automatic audio indexing systems.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Electrical Engineering

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2015

Author dc:creator

• Li, Danfeng

Contributors dc:contributor

• Levinson, Stephen E.

• Computer Science

Language dc:language

eng

Identifier

(MiAaPQ)AAI3086118

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/80822