Computational Approaches to Facilitate Automated Interchange between Music and Art

<p>Recently, there has been a tremendous increase in generating and synthesizing music and art using various computational techniques. An area that is still under-researched, however, is how one medium can be converted into the other, while maintaining the overall aesthetics. Over the last few centuries, artists, composers, and scholars, have attempted to use substitute one form of art for the other: by proposing techniques where music notes are synonymous to colors, by inventing instruments that combine the aesthetics of music and visual art, and by incorporating the two media in live performances. A widely accepted computational approach, for the conversion, has yet to be introduced and there is some space for experimentation: to develop and polish a mechanism that can be adopted globally. This dissertation explores different computational techniques that can be used to convert one medium into the other, with a varying human input component. First, a simple computational approach to convert piano compositions into paintings is discussed that relies heavily on human encoded metrics to perform the conversion. Next, the embedding layer of a simple neural network is explored to provide information about artists and their artworks. The idea of using embedding layers for data compression is extended to produce a pure computational approach, where latent spaces of two trained variational autoencoder networks are interchanged. Finally, the earlier approaches are merged to explore human encoded metrics mapped to a generative model to interpolate music synthesis. For all approaches, results are derived to further enhance the commentary on the interchange of music and art.</p>

Name thesis:degree_name

Doctor of Philosophy (PhD)

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computational and Data Sciences

Year dc:date.available

2022

Author dc:creator

• Ali, Rao Hamza

Contributors dc:contributor

• Dr. Erik Linstead
• Dr. Elizabeth Stevens
• Dr. Grace Fong

• machine learning
• neural networks
• computational systems
• music
• art
• Artificial Intelligence and Robotics
• Data Science
• Other Computer Sciences

Repository record dc:identifier

https://digitalcommons.chapman.edu/cads_dissertations/28

OAI identifier oai:identifier

oai:digitalcommons.chapman.edu:cads_dissertations-1028