Experiential Graphic Sonification for Visual and Auditory Communication Design and Musical Expression

This dissertation explains two sonification platforms designed for image-sound association study and art, and the study results. A platform for user study was developed first and an artistic audiovisual platform was derived based on it. First, the five image-sound association studies were conducted to see whether people can successfully associate sounds and fundamental shapes (i.e., a circle, a triangle, a square, lines, curves, and other custom shapes) and the correct answer rate was high. Then, the same sonification platform was transformed by adding colors to the audiovisual platform for artistic/musical expression. A line-by-line sonification method and an object-oriented method were newly developed to sonify the background and the shapes separately. To enhance user experience, the sound of each shape was spatialized in a multi-layer speaker environment or a virtual listening environment.

Name thesis:degree_name

Doctor of Philosophy

Level thesis:degree_level

doctoral

Discipline thesis:degree_discipline

Individual Interdisciplinary PhD

Department dc:contributor.department

Human Development

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

2022

Author dc:creator

• Joo, Woohun

Chair dc:contributor.committeechair

• Nichols, Charles Sabin

Committee members dc:contributor.committeemember

• Duer, Zachary R.
• Jeon, Myounghoon
• Finley, Patrick

• human computer interaction
• sonification
• communication design
• interaction design
• sound art
• visual arts
• visualization

Statement dc:rights

• Creative Commons Attribution-ShareAlike 4.0 International

Licence dc:rights.uri

• http://creativecommons.org/licenses/by-sa/4.0/

Language dc:language.iso

en

Dc Identifier Other

vt_gsexam:35012

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/110969