{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/129863"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/129863","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Fine-grained error analysis in machine translation","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-20 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-20 without embargo terms","abstract_has_math":false,"creators":["Dell, Brennan"],"institution":"University of Illinois Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Linguistics","degree_department":null,"school":null,"contributors":["Dunn, Jonathan","Yoon, James","Maskharashvili, Aleksandre","Tang, Yan"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-07-16","date_published":"2025-07-16","updated_at":"2026-07-22T22:25:06Z","subjects":["Machine Translation","Computational Linguistics","Turkish","Finnish","Evidentiality"],"languages":["en","eng"],"rights":["Copyright 2025 Brennan Dell"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/129863","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Dunn, Jonathan","Yoon, James","Maskharashvili, Aleksandre","Tang, Yan"]},{"key":"dc:creator","label":"Author","values":["Dell, Brennan"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-07-16","2025-08"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Linguistics"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Machine Translation","Computational Linguistics","Turkish","Finnish","Evidentiality"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2025 Brennan Dell"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/129863"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-20 without embargo terms","The student, Brennan Dell, accepted the attached license on 2025-07-11 at 01:55.","The student, Brennan Dell, submitted this Dissertation for approval on 2025-07-11 at 02:02.","This Dissertation was approved for publication on 2025-07-16 at 13:24.","DSpace SAF Submission Ingestion Package generated from Vireo submission #22500 on 2025-10-20 at 16:57:47","This dissertation focuses on the fine-grained evaluation method for machine translation. We define fine-grained tests as methods which evaluate translation quality with respect to a particular phenomenon. These methods contrast with one-dimensional evaluation methods, which quantify translation quality as a single number. The BLEU score, and more recent neural translation metrics like XCOMET, are examples of such metrics. In this work we demonstrate, from our own experiments and from a review of related work, that fine-grained evaluation methods are more useful tools for translation evaluation. We argue that this is the case because translation quality is inherently multidimensional. A correct translation must not only convey the meaning of the source language, but must also be grammatical in the target language. These two factors themselves are complex phenomena, with many orthogonal components. Fine-grained tests are thus well-suited to the fundamentally multidimensional nature of translation quality. In our first experiment, we demonstrate that fine-grained tests are uniquely useful for understanding how the training neural machine translation system unfolds. Additionally, through a preliminary experiment targeting the translation of the Turkish evidentiality morpheme, we observe translation system behaviors that suggest they may not have acquired evidentiality. These findings, which we were not able to explain with linguistic or sentence complexity features, motivate further research on this phenomenon. We present this work as a review of related work on fine-grained error analysis in machine translation, supplemented with our own findings, analysis, and recommendations. We do this to aid the future development of such tests, which we believe will be crucial for navigating our current era which is increasingly dominated by large language models."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Fine-grained error analysis in machine translation"]}]}],"canonical_facts":{"dc:contributor":["Dunn, Jonathan","Yoon, James","Maskharashvili, Aleksandre","Tang, Yan"],"dc:creator":["Dell, Brennan"],"dc:date":["2025-07-16","2025-08"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-20 without embargo terms","The student, Brennan Dell, accepted the attached license on 2025-07-11 at 01:55.","The student, Brennan Dell, submitted this Dissertation for approval on 2025-07-11 at 02:02.","This Dissertation was approved for publication on 2025-07-16 at 13:24.","DSpace SAF Submission Ingestion Package generated from Vireo submission #22500 on 2025-10-20 at 16:57:47","This dissertation focuses on the fine-grained evaluation method for machine translation. We define fine-grained tests as methods which evaluate translation quality with respect to a particular phenomenon. These methods contrast with one-dimensional evaluation methods, which quantify translation quality as a single number. The BLEU score, and more recent neural translation metrics like XCOMET, are examples of such metrics. In this work we demonstrate, from our own experiments and from a review of related work, that fine-grained evaluation methods are more useful tools for translation evaluation. We argue that this is the case because translation quality is inherently multidimensional. A correct translation must not only convey the meaning of the source language, but must also be grammatical in the target language. These two factors themselves are complex phenomena, with many orthogonal components. Fine-grained tests are thus well-suited to the fundamentally multidimensional nature of translation quality. In our first experiment, we demonstrate that fine-grained tests are uniquely useful for understanding how the training neural machine translation system unfolds. Additionally, through a preliminary experiment targeting the translation of the Turkish evidentiality morpheme, we observe translation system behaviors that suggest they may not have acquired evidentiality. These findings, which we were not able to explain with linguistic or sentence complexity features, motivate further research on this phenomenon. We present this work as a review of related work on fine-grained error analysis in machine translation, supplemented with our own findings, analysis, and recommendations. We do this to aid the future development of such tests, which we believe will be crucial for navigating our current era which is increasingly dominated by large language models."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/129863"],"dc:language":["en","eng"],"dc:rights":["Copyright 2025 Brennan Dell"],"dc:subject":["Machine Translation","Computational Linguistics","Turkish","Finnish","Evidentiality"],"dc:title":["Fine-grained error analysis in machine translation"],"dc:type":["text"],"thesis:degree_discipline":["Linguistics"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:06Z"}