The benefits of acoustic perceptual information for speech processing systems

The frame-synchronized framework has dominated many speech processing systems, such as ASR and AED targeting human speech activities. These systems have little consideration for the science behind speech and treat the task as a simple statistical classification. The framework also assumes each feature vector to be equally important to the task. However, through some preliminary experiments, this study has found evidence that some concepts defined in speech perception theories such as auditory roughness and acoustic landmarks can act as heuristics to these systems and benefit them in multiple ways. Findings of acoustic landmarks hint that the idea of treating each frame equally might not be optimal. In some cases, landmark information can improve system accuracy through highlighting the more significant frames, or improve the acoustic model accuracy by training through MTL. Further investigation into the topic found experimental evidence suggesting that acoustic landmark information can also benefit end-to-end acoustic models trained through CTC loss. With the help of acoustic landmarks, CTC models can converge with less training data and achieve lower error rate. For the first time, positive results were collected on a mid-size ASR corpus (WSJ) for acoustic landmarks. The results indicate that audio perception information can benefit a broad range of audio processing systems.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Electrical & Computer Engr

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2019

Author dc:creator

• He, Di

Contributors dc:contributor

• Chen, Deming
• Hasegawa-Johnson, Mark
• Wong, Martin
• Lim, Boon Pang

• ASR
• AED
• Acoustic Landmark
• Auditory Roughness
• IoT
• FPGA
• MTL
• CTC

Statement dc:rights

• Copyright 2019 Di He

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/104888

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/104888