Intuitive Audio Interaction and Control in Multi-Source Environments

In an increasingly noisy world, managing auditory focus is a persistent challenge. This thesis explores how embodied interactions—primarily head tracking, alongside experiments with gaze tracking, speech commands, and audio-visual segmentation—can enhance user control over complex auditory environments. By linking head orientation to volume adjustments, we investigated whether natural, instinctive movements could serve as intuitive, hands-free mechanisms for isolating and amplifying relevant sounds. User studies revealed that head tracking is effective in structured audio contexts, such as music, where distinct sources are easily separable. However, its utility diminishes in dense, overlapping conversations, highlighting the need for finer control mechanisms. While gaze and segmentation offer promising refinements, cognitive load and system responsiveness remain key challenges. These findings underscore that embodied audio interaction must be adaptive, content-aware, and seamlessly integrated with user intent.This research contributes to human-computer interaction by demonstrating both the potential and limitations of movement-based audio control. Future work should refine multimodal fusion, improve segmentation accuracy, and enhance accessibility to create systems that dynamically respond to users’ natural behaviors—reducing cognitive strain and enabling more fluid, user-centric auditory experiences.

Name thesis:degree_name

Master

Department dc:contributor.department

Program in Media Arts and Sciences (Massachusetts Institute of Technology)

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2025

Author dc:creator

• Oduniyi, Erick O.

Advisor dc:contributor.advisor

• Lippman, Andrew B.

Statement dc:rights

• In Copyright - Educational Use Permitted
• Copyright retained by author(s)

Licence dc:rights.uri

• https://rightsstatements.org/page/InC-EDU/1.0/

Handle dc:identifier.uri

https://hdl.handle.net/1721.1/159149

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/159149