{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/89050"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/89050","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Machine learning approaches to improving mispronunciation detection on an imbalanced corpus","abstract":"This thesis reports the investigations into the task of phone-level pronunciation error detection, the performance of which is heavily affected by the imbalanced distribution of the classes in a manually annotated data set of non-native English (Read Aloud responses from the TOEFL Junior Pilot assessment). In order to address problems caused by this extreme class imbalance, two machine learning approaches, cost-sensitive learning and over-sampling, are explored to improve the classification performance. Specifically, approaches which assigned weights inversely proportional to class frequencies and synthetic minority over-sampling technique (SMOTE) were applied to a range of classifiers using feature sets that included information about the acoustic signal, the linguistic properties of the utterance, and word identity. Empirical experiments demonstrate that both balancing approaches lead to a substantial performance improvement (in terms of f1 score) over the baseline on this extremely imbalanced data set. In addition, this thesis also discusses which features are the most important and which classifiers are most effective for the task of identifying phone-level pronunciation errors in non-native speech.","abstract_html":"This thesis reports the investigations into the task of phone-level pronunciation error detection, the performance of which is heavily affected by the imbalanced distribution of the classes in a manually annotated data set of non-native English (Read Aloud responses from the TOEFL Junior Pilot assessment). In order to address problems caused by this extreme class imbalance, two machine learning approaches, cost-sensitive learning and over-sampling, are explored to improve the classification performance. Specifically, approaches which assigned weights inversely proportional to class frequencies and synthetic minority over-sampling technique (SMOTE) were applied to a range of classifiers using feature sets that included information about the acoustic signal, the linguistic properties of the utterance, and word identity. Empirical experiments demonstrate that both balancing approaches lead to a substantial performance improvement (in terms of f1 score) over the baseline on this extremely imbalanced data set. In addition, this thesis also discusses which features are the most important and which classifiers are most effective for the task of identifying phone-level pronunciation errors in non-native speech.","abstract_has_math":false,"creators":["Yang, Xuesong"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Electrical & Computer Engineering","degree_department":null,"school":null,"contributors":["Hasegawa-Johnson, Mark A."],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2016,"date_issued":"2016-03-02T19:34:21Z","date_published":"2016-03-02T19:34:21Z","updated_at":"2026-07-22T22:26:32Z","subjects":["Imbalanced Learning","Sampling Methods","Pronunciation Error Detection","Spoken Language Assessment","Computer Assisted Language Learning"],"languages":["en"],"rights":["Copyright 2015 Xuesong Yang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/89050","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Hasegawa-Johnson, Mark A."]},{"key":"dc:creator","label":"Author","values":["Yang, Xuesong"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2016-03-02T19:34:21Z","2018-03-28T09:15:32Z","2015-12-07","2015-12"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Imbalanced Learning","Sampling Methods","Pronunciation Error Detection","Spoken Language Assessment","Computer Assisted Language Learning"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2015 Xuesong Yang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/89050"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["This thesis reports the investigations into the task of phone-level pronunciation error detection, the performance of which is heavily affected by the imbalanced distribution of the classes in a manually annotated data set of non-native English (Read Aloud responses from the TOEFL Junior Pilot assessment). In order to address problems caused by this extreme class imbalance, two machine learning approaches, cost-sensitive learning and over-sampling, are explored to improve the classification performance. Specifically, approaches which assigned weights inversely proportional to class frequencies and synthetic minority over-sampling technique (SMOTE) were applied to a range of classifiers using feature sets that included information about the acoustic signal, the linguistic properties of the utterance, and word identity. Empirical experiments demonstrate that both balancing approaches lead to a substantial performance improvement (in terms of f1 score) over the baseline on this extremely imbalanced data set. In addition, this thesis also discusses which features are the most important and which classifiers are most effective for the task of identifying phone-level pronunciation errors in non-native speech.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2016-03-02 without embargo terms","The student, Xuesong Yang, accepted the attached license on 2015-12-04 at 11:53.","The student, Xuesong Yang, submitted this Thesis for approval on 2015-12-04 at 12:05.","This Thesis was approved for publication on 2015-12-07 at 11:50.","DSpace SAF Submission Ingestion Package generated from Vireo submission #8935 on 2016-03-02 at 12:51:28","Made available in DSpace on 2016-03-02T19:34:21Z (GMT). No. of bitstreams: 2 YANG-THESIS-2015.pdf: 761495 bytes, checksum: 1d0976ea84c7cbc98ed09c3931e9cbbd (MD5) LICENSE.txt: 4209 bytes, checksum: 5d21edcb5fb410adccbd229350336fe3 (MD5) Previous issue date: 2015-12-07","Limited Restriction Lifted for Item 91251 on 2018-03-28T09:15:32Z."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Machine learning approaches to improving mispronunciation detection on an imbalanced corpus"]}]}],"canonical_facts":{"dc:contributor":["Hasegawa-Johnson, Mark A."],"dc:creator":["Yang, Xuesong"],"dc:date":["2016-03-02T19:34:21Z","2018-03-28T09:15:32Z","2015-12-07","2015-12"],"dc:description":["This thesis reports the investigations into the task of phone-level pronunciation error detection, the performance of which is heavily affected by the imbalanced distribution of the classes in a manually annotated data set of non-native English (Read Aloud responses from the TOEFL Junior Pilot assessment). In order to address problems caused by this extreme class imbalance, two machine learning approaches, cost-sensitive learning and over-sampling, are explored to improve the classification performance. Specifically, approaches which assigned weights inversely proportional to class frequencies and synthetic minority over-sampling technique (SMOTE) were applied to a range of classifiers using feature sets that included information about the acoustic signal, the linguistic properties of the utterance, and word identity. Empirical experiments demonstrate that both balancing approaches lead to a substantial performance improvement (in terms of f1 score) over the baseline on this extremely imbalanced data set. In addition, this thesis also discusses which features are the most important and which classifiers are most effective for the task of identifying phone-level pronunciation errors in non-native speech.","Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2016-03-02 without embargo terms","The student, Xuesong Yang, accepted the attached license on 2015-12-04 at 11:53.","The student, Xuesong Yang, submitted this Thesis for approval on 2015-12-04 at 12:05.","This Thesis was approved for publication on 2015-12-07 at 11:50.","DSpace SAF Submission Ingestion Package generated from Vireo submission #8935 on 2016-03-02 at 12:51:28","Made available in DSpace on 2016-03-02T19:34:21Z (GMT). No. of bitstreams: 2 YANG-THESIS-2015.pdf: 761495 bytes, checksum: 1d0976ea84c7cbc98ed09c3931e9cbbd (MD5) LICENSE.txt: 4209 bytes, checksum: 5d21edcb5fb410adccbd229350336fe3 (MD5) Previous issue date: 2015-12-07","Limited Restriction Lifted for Item 91251 on 2018-03-28T09:15:32Z."],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/89050"],"dc:language":["en"],"dc:rights":["Copyright 2015 Xuesong Yang"],"dc:subject":["Imbalanced Learning","Sampling Methods","Pronunciation Error Detection","Spoken Language Assessment","Computer Assisted Language Learning"],"dc:title":["Machine learning approaches to improving mispronunciation detection on an imbalanced corpus"],"dc:type":["text"],"thesis:degree_discipline":["Electrical & Computer Engineering"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:26:32Z"}