Dynamic Bayesian Networks for Information Fusion With Applications to Human-Computer Interfaces

Recent advances in various display and virtual technologies coupled with an explosion in available computing power have given rise to a number of novel human-computer interaction (HCI) modalities---speech, vision-based gesture recognition, eye tracking, EEG, etc. However, despite the abundance of novel interaction devices, the naturalness and efficiency of HCI has remained low. This is due in particular to the lack of robust sensory data interpretation techniques. To deal with the task of interpreting single and multiple interaction modalities this dissertation establishes a novel probabilistic approach based on dynamic Bayesian networks (DBNs). As a generalization of the successful hidden Markov models, DBNs are a natural basis for the general temporal action interpretation task. The problem of interpretation of single or multiple interacting modalities can then be viewed as a Bayesian inference task. In this work three complex DBN models are introduced: mixtures of DBNs, mixed-state DBNs, and coupled HMMs. In-depth study of these models yields efficient approximate inference and parameter learning techniques applicable to a wide variety of problems. Experimental validation of the proposed approaches in the domains of gesture and speech recognition confirms the model's applicability to both unimodal and multimodal interpretation tasks.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Electrical Engineering

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2015

Author dc:creator

• Pavlovic, Vladimir Ivan

Contributors dc:contributor

• Huang, Thomas S.

• Computer Science

Language dc:language

eng

Identifier

(MiAaPQ)AAI9921722

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/81285