{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/108196"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/108196","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Semi-supervised cycle-consistency training for end-to-end ASR using unpaired speech","abstract":"The thesis is a replication of the work by Takaaki Hori and his colleagues (2019), which introduces a new method to train end-to-end automatic speech recognition (ASR) models using unpaired speech. In general, large amounts of paired data (speech and text) are needed to train an end-to-end automatic speech recognition system. To alleviate the problem of limited paired data, the idea of cycle-consistency losses has been proposed recently in areas such as machine translation and computer vision. In ASR, cycle-consistency training is achieved by building a reverse system, e.g., a text-to-speech system, and designing a loss based on the reconstructed signal and the original one. However, it is not straightforward to apply cycle-consistency in ASR as information would be lost in the text bottleneck. Tomoki Hayashi et al. (2018) tackled this problem via a text-to-encoder (TTE) model, which predicts encoder states extracted by a pre-trained end-to-end ASR encoder from text input. In this work, the TTE model was used as the reverse system and a loss was defined by comparing the original ASR encoder states and the reconstructed encoder states from the TTE model. Using encoder states instead of raw acoustic features as targets, the model can learn attention much faster and avoid the modeling of speaker dependencies. Our experimental results on the LibriSpeech corpus were similar to the results of Hori et al. The initial ASR and TTE models were trained with LibriSpeech 100-hour paired speech data. By applying cycle-consistency loss and retraining the speech-to-text-to-encoder chain model using one third of LibriSpeech 360-hour unpaired speech data, ASR word error rate was reduced from 25.8% to 21.7% on the LibriSpeech 5-hour test data.","abstract_html":"The thesis is a replication of the work by Takaaki Hori and his colleagues (2019), which introduces a new method to train end-to-end automatic speech recognition (ASR) models using unpaired speech. In general, large amounts of paired data (speech and text) are needed to train an end-to-end automatic speech recognition system. To alleviate the problem of limited paired data, the idea of cycle-consistency losses has been proposed recently in areas such as machine translation and computer vision. In ASR, cycle-consistency training is achieved by building a reverse system, e.g., a text-to-speech system, and designing a loss based on the reconstructed signal and the original one. However, it is not straightforward to apply cycle-consistency in ASR as information would be lost in the text bottleneck. Tomoki Hayashi et al. (2018) tackled this problem via a text-to-encoder (TTE) model, which predicts encoder states extracted by a pre-trained end-to-end ASR encoder from text input. In this work, the TTE model was used as the reverse system and a loss was defined by comparing the original ASR encoder states and the reconstructed encoder states from the TTE model. Using encoder states instead of raw acoustic features as targets, the model can learn attention much faster and avoid the modeling of speaker dependencies. Our experimental results on the LibriSpeech corpus were similar to the results of Hori et al. The initial ASR and TTE models were trained with LibriSpeech 100-hour paired speech data. By applying cycle-consistency loss and retraining the speech-to-text-to-encoder chain model using one third of LibriSpeech 360-hour unpaired speech data, ASR word error rate was reduced from 25.8% to 21.7% on the LibriSpeech 5-hour test data.","abstract_has_math":false,"creators":["Wu, Ningkai"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Electrical & Computer Engr","degree_department":null,"school":null,"contributors":["Hasegawa-Johnson, Mark"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2020,"date_issued":"2020-08-26T23:58:49Z","date_published":"2020-08-26T23:58:49Z","updated_at":"2026-07-22T22:24:48Z","subjects":["Speech recognition","Semi-supervised training"],"languages":["en"],"rights":["Copyright 2020 Ningkai Wu"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/108196","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Hasegawa-Johnson, Mark"]},{"key":"dc:creator","label":"Author","values":["Wu, Ningkai"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2020-08-26T23:58:49Z","2022-08-26T23:58:55Z","2020-05-14","2020-05"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engr"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Speech recognition","Semi-supervised training"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2020 Ningkai Wu"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/108196"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["The thesis is a replication of the work by Takaaki Hori and his colleagues (2019), which introduces a new method to train end-to-end automatic speech recognition (ASR) models using unpaired speech. In general, large amounts of paired data (speech and text) are needed to train an end-to-end automatic speech recognition system. To alleviate the problem of limited paired data, the idea of cycle-consistency losses has been proposed recently in areas such as machine translation and computer vision. In ASR, cycle-consistency training is achieved by building a reverse system, e.g., a text-to-speech system, and designing a loss based on the reconstructed signal and the original one. However, it is not straightforward to apply cycle-consistency in ASR as information would be lost in the text bottleneck. Tomoki Hayashi et al. (2018) tackled this problem via a text-to-encoder (TTE) model, which predicts encoder states extracted by a pre-trained end-to-end ASR encoder from text input. In this work, the TTE model was used as the reverse system and a loss was defined by comparing the original ASR encoder states and the reconstructed encoder states from the TTE model. Using encoder states instead of raw acoustic features as targets, the model can learn attention much faster and avoid the modeling of speaker dependencies. Our experimental results on the LibriSpeech corpus were similar to the results of Hori et al. The initial ASR and TTE models were trained with LibriSpeech 100-hour paired speech data. By applying cycle-consistency loss and retraining the speech-to-text-to-encoder chain model using one third of LibriSpeech 360-hour unpaired speech data, ASR word error rate was reduced from 25.8% to 21.7% on the LibriSpeech 5-hour test data.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2022-05-01","The student, Ningkai Wu, accepted the attached license on 2020-05-13 at 08:59.","The student, Ningkai Wu, submitted this Thesis for approval on 2020-05-13 at 09:52.","This Thesis was approved for publication on 2020-05-14 at 16:32.","DSpace SAF Submission Ingestion Package generated from Vireo submission #15372 on 2020-08-25 at 17:31:22","Made available in DSpace on 2020-08-26T23:58:49Z (GMT). No. of bitstreams: 2 WU-THESIS-2020.pdf: 509349 bytes, checksum: 22d2eb1b6be3a4e825a70718bd67c2ea (MD5) LICENSE.txt: 4207 bytes, checksum: 2ee348ece1c52346e89ff5f7857c382f (MD5) Previous issue date: 2020-05-14","Embargo set by: Seth Robbins for item 115809 Lift date: 2022-08-26T23:58:55Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only"]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Semi-supervised cycle-consistency training for end-to-end ASR using unpaired speech"]}]}],"canonical_facts":{"dc:contributor":["Hasegawa-Johnson, Mark"],"dc:creator":["Wu, Ningkai"],"dc:date":["2020-08-26T23:58:49Z","2022-08-26T23:58:55Z","2020-05-14","2020-05"],"dc:description":["The thesis is a replication of the work by Takaaki Hori and his colleagues (2019), which introduces a new method to train end-to-end automatic speech recognition (ASR) models using unpaired speech. In general, large amounts of paired data (speech and text) are needed to train an end-to-end automatic speech recognition system. To alleviate the problem of limited paired data, the idea of cycle-consistency losses has been proposed recently in areas such as machine translation and computer vision. In ASR, cycle-consistency training is achieved by building a reverse system, e.g., a text-to-speech system, and designing a loss based on the reconstructed signal and the original one. However, it is not straightforward to apply cycle-consistency in ASR as information would be lost in the text bottleneck. Tomoki Hayashi et al. (2018) tackled this problem via a text-to-encoder (TTE) model, which predicts encoder states extracted by a pre-trained end-to-end ASR encoder from text input. In this work, the TTE model was used as the reverse system and a loss was defined by comparing the original ASR encoder states and the reconstructed encoder states from the TTE model. Using encoder states instead of raw acoustic features as targets, the model can learn attention much faster and avoid the modeling of speaker dependencies. Our experimental results on the LibriSpeech corpus were similar to the results of Hori et al. The initial ASR and TTE models were trained with LibriSpeech 100-hour paired speech data. By applying cycle-consistency loss and retraining the speech-to-text-to-encoder chain model using one third of LibriSpeech 360-hour unpaired speech data, ASR word error rate was reduced from 25.8% to 21.7% on the LibriSpeech 5-hour test data.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2022-05-01","The student, Ningkai Wu, accepted the attached license on 2020-05-13 at 08:59.","The student, Ningkai Wu, submitted this Thesis for approval on 2020-05-13 at 09:52.","This Thesis was approved for publication on 2020-05-14 at 16:32.","DSpace SAF Submission Ingestion Package generated from Vireo submission #15372 on 2020-08-25 at 17:31:22","Made available in DSpace on 2020-08-26T23:58:49Z (GMT). No. of bitstreams: 2 WU-THESIS-2020.pdf: 509349 bytes, checksum: 22d2eb1b6be3a4e825a70718bd67c2ea (MD5) LICENSE.txt: 4207 bytes, checksum: 2ee348ece1c52346e89ff5f7857c382f (MD5) Previous issue date: 2020-05-14","Embargo set by: Seth Robbins for item 115809 Lift date: 2022-08-26T23:58:55Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only"],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/108196"],"dc:language":["en"],"dc:rights":["Copyright 2020 Ningkai Wu"],"dc:subject":["Speech recognition","Semi-supervised training"],"dc:title":["Semi-supervised cycle-consistency training for end-to-end ASR using unpaired speech"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Electrical & Computer Engr"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:48Z"}