Assessing the Quality of Synthetic Speech when using Enhanced Speech as Training Data
type: CreativeWork
description: Both speech synthesis and speech enhancement are well researched fields, but their interaction remains under-explored. In particular, the effectiveness of using enhanced speech to train a speech synthesis model is still relatively unknown. This thesis investigates the effects of using enhanced speech to train a speech synthesis model on the overall quality of the synthesised speech, and in doing so, gain a better understanding of the interactions between different speech enhancement algorithms, speech synthesis architectures, gender of the voice and the noise types that degrade the speech used to train the speech synthesis model. A series of large-scale perception tests were conducted whereby over 100 participants evaluated the quality of utterances generated by speech synthesis model trained on noisy or enhanced speech. For this study, 160 synthetic voices were created, one for every combination of four speech enhancement methods, four noise types at two SNRs, two voices, one male and one female for two speech synthesis models – MaryTTS and Tacotron 2. Using linear mixed effect models, the study discerned the interactions between speech enhancement algorithms, speech synthesis architectures, gender of the voice and noise types. The results from this analysis suggest that different genders of voices resulted in similar trends depending on the speech enhancement methods, speech synthesis methods and noise types. However, the speech synthesis methods were affected differently by the various speech enhancement methods and noise types. An investigation into evaluating the quality of synthesised voices created from noisy or enhanced speech using eleven objective quality metrics was also conducted. This investigation found that several common objective quality metrics used to evaluate speech enhancement also have relatively high correlations with subjective quality of utterances generated by speech synthesisers trained on noisy or enhanced speech.
abstract: Both speech synthesis and speech enhancement are well researched fields, but their interaction remains under-explored. In particular, the effectiveness of using enhanced speech to train a speech synthesis model is still relatively unknown. This thesis investigates the effects of using enhanced speech to train a speech synthesis model on the overall quality of the synthesised speech, and in doing so, gain a better understanding of the interactions between different speech enhancement algorithms, speech synthesis architectures, gender of the voice and the noise types that degrade the speech used to train the speech synthesis model. A series of large-scale perception tests were conducted whereby over 100 participants evaluated the quality of utterances generated by speech synthesis model trained on noisy or enhanced speech. For this study, 160 synthetic voices were created, one for every combination of four speech enhancement methods, four noise types at two SNRs, two voices, one male and one female for two speech synthesis models – MaryTTS and Tacotron 2. Using linear mixed effect models, the study discerned the interactions between speech enhancement algorithms, speech synthesis architectures, gender of the voice and noise types. The results from this analysis suggest that different genders of voices resulted in similar trends depending on the speech enhancement methods, speech synthesis methods and noise types. However, the speech synthesis methods were affected differently by the various speech enhancement methods and noise types. An investigation into evaluating the quality of synthesised voices created from noisy or enhanced speech using eleven objective quality metrics was also conducted. This investigation found that several common objective quality metrics used to evaluate speech enhancement also have relatively high correlations with subjective quality of utterances generated by speech synthesisers trained on noisy or enhanced speech.
author: Eng, Nicholas
keywords: Speech Enhancement, Speech Synthesis, Text-To-Speech, Audio Quality, Speech Perception, Speech Quality Assessment, Objective Metrics, Deep Learning
license: https://researchspace.auckland.ac.nz/docs/uoa-docs/rights.htm
date published: 2025
date modified: 2026-07-24T01:06:29Z
provider: University of Auckland
publisher: Networked Digital Library of Theses and Dissertations
type: WebPage
source organization: ResearchSpace@Auckland

PropertyValue
property ID: Handle
value: https://hdl.handle.net/2292/74747