Towards an end-to-end music transcription system using neural networks

Transcription is the task of writing down instructions on how to play a particular piece of music, including individual notes, note durations, embellishments and so on. While most major works in the traditional repertoire have readily available transcriptions for various instrument arrangements, this is not as common in genres where improvisation is more prevalent, such as Jazz, or where the piece has a very particular purpose, as in motion picture and video game soundtracks. It has notable parallels with the task of Automatic Speech Recognition (ASR) and indeed from this connection arises some natural Machine Learning-based approaches. However, these methods usually involve carefully designed preprocessing steps, or transcription into less flexible representations, such as piano rolls, which are harder to read for humans. This work investigates the feasibility of designing an end-to-end music transcription system that takes in raw audio recordings and produces Lilypond notation, which can directly generate easily-recognizable sheet music. In keeping with modern ASR methods, this task is modeled as a sequence-to-sequence problem using Convolutional and Recurrent Neural Networks. The system is shown to perform well for both monophonic (single melody on a single instrument) and polyphonic music (parallel melodies on possibly different instruments) for randomly generated pieces played by the piano and various other common orchestra instruments.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2018

Author dc:creator

• Correa Carvalho, Ralf Gunter

Contributors dc:contributor

• Smaragdis, Paris

• Automatic music transcription
• Machine learning
• Deep learning
• Signal processing

Statement dc:rights

• Copyright 2017 Ralf Gunter Correa Carvalho

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/99126

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/99126