The Effect of Sonification in Informal Learning: From Fairy Tales to Art Galleries

As technology continues to advance in the present era, lifelong learning has become essential across all stages of life. Learning occurs not only in formal institutional settings but also in informal contexts embedded in daily life. Within these informal learning environments, sonification can play a pivotal role by facilitating learning through auditory interaction, interpretation, and engagement. However, the effectiveness of sonification can vary based on contextual factors and individual capabilities. To investigate the potential of sonification in informal learning, this dissertation presents four empirical studies across two domains: human-robot interaction (HRI) and art appreciation in gallery settings. The first study examined how emotion-reflecting sonification, compared to emotion mitigating sonification, influences the experience of listening to fairy tales, focusing on participants' enjoyment, empathy, and immersion. The second study compared sentiment based sonification with classical music in robot storytelling, evaluating their impact on comprehension, emotional interpretation, and engagement. The third study investigated the role of Augmented Audio in an art gallery environment, assessing its influence on participants' emotional responses and engagement during art appreciation. The fourth study explored how Augmented Audio and Augmented Visual elements influence comprehension, emotional reaction, and sustained engagement with artworks. Collectively, this dissertation offers insights into how sonification can be effectively applied across varied informal learning contexts, from narrative storytelling to visual art appreciation, to foster immersive, emotionally resonant, and cognitively enriching educational experiences.

Name thesis:degree_name

Doctor of Philosophy

Level thesis:degree_level

doctoral

Discipline thesis:degree_discipline

Industrial and Systems Engineering

Department dc:contributor.department

Industrial and Systems Engineering

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

2025

Author dc:creator

• Lee, Yeaji

Chair dc:contributor.committeechair

• Jeon, Myounghoon

Committee members dc:contributor.committeemember

• Lee, Sang Won
• Dickerson, Deborah Elspeth
• Roofigari-Esfahan, Nazila

• sonification
• informal learning

Statement dc:rights

• In Copyright

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Language dc:language.iso

en

Dc Identifier Other

vt_gsexam:44530

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/137624