{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/50728"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/50728","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Raters' different judgments of intelligibility in the English placement test at University of Illinois at Urbana-Champaign","abstract":"This study investigated raters’ judgments of test-takers’ intelligibility on Phase I of the EPT (English Placement Test), and how their various teaching experiences and different native languages affected their judgments. The EPT is administered to incoming international students whose TOEFL scores do not meet campus or departmental cutoff scores at the University of Illinois at Urbana Champaign. Phase I of the EPT oral section is designed to measure students’ intelligibility, and employs a group discussion format. Nine raters with different teaching experience and native languages were interviewed, and their responses were analyzed qualitatively. Results show that all the raters interpreted oral intelligibility differentially, and raters’ different understanding of the constructs underlying intelligibility resulted in different rating scales. This result wasn’t affected by raters’ teaching experience or language backgrounds. Rather than these, familiarity with a specific accent or rater’s individual ideas about intelligibility had more influence on their judgment. When they were asked about which specific features contribute more to intelligibility, most raters responded that segmental factors and lexical stress were important factors for their judgment. This might be because English Placement Test is a tool for assessing the ‘word level’ intelligibility. Phase I of the English Placement Test uses a one-on-one interview format or a group-administered format at raters’ discretion. Most raters reported that they prefer the one-on-one interview format. They argued that even though the group-administered format had the benefit of reducing time and expense, it would threaten the validity of the test because of the possibility of copying another test taker’s response or of getting nervous due to the situation in which test takers had to speak in front of group members. Providing a clarified and unified definition of intelligibility in the recalibration session is suggested. Additionally, raters still need to have sufficient knowledge of what the group oral discussion format is and know how to manage this format.","abstract_html":"This study investigated raters’ judgments of test-takers’ intelligibility on Phase I of the EPT (English Placement Test), and how their various teaching experiences and different native languages affected their judgments. The EPT is administered to incoming international students whose TOEFL scores do not meet campus or departmental cutoff scores at the University of Illinois at Urbana Champaign. Phase I of the EPT oral section is designed to measure students’ intelligibility, and employs a group discussion format. Nine raters with different teaching experience and native languages were interviewed, and their responses were analyzed qualitatively. Results show that all the raters interpreted oral intelligibility differentially, and raters’ different understanding of the constructs underlying intelligibility resulted in different rating scales. This result wasn’t affected by raters’ teaching experience or language backgrounds. Rather than these, familiarity with a specific accent or rater’s individual ideas about intelligibility had more influence on their judgment. When they were asked about which specific features contribute more to intelligibility, most raters responded that segmental factors and lexical stress were important factors for their judgment. This might be because English Placement Test is a tool for assessing the ‘word level’ intelligibility. Phase I of the English Placement Test uses a one-on-one interview format or a group-administered format at raters’ discretion. Most raters reported that they prefer the one-on-one interview format. They argued that even though the group-administered format had the benefit of reducing time and expense, it would threaten the validity of the test because of the possibility of copying another test taker’s response or of getting nervous due to the situation in which test takers had to speak in front of group members. Providing a clarified and unified definition of intelligibility in the recalibration session is suggested. Additionally, raters still need to have sufficient knowledge of what the group oral discussion format is and know how to manage this format.","abstract_has_math":false,"creators":["Lee, Ji Eun"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.A.","degree_level":"Thesis","degree_discipline":"Teaching of English Sec Lang","degree_department":null,"school":null,"contributors":["Davidson, Frederick G."],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2014,"date_issued":"2014-09-16T17:25:56Z","date_published":"2014-09-16T17:25:56Z","updated_at":"2026-07-22T22:25:41Z","subjects":["English Placement Test (EPT)","Language Testing","Intelligilibility","Group oral format"],"languages":["en"],"rights":["Copyright 2014 Ji Eun Lee"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/50728","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Davidson, Frederick G."]},{"key":"dc:creator","label":"Author","values":["Lee, Ji Eun"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2014-09-16T17:25:56Z","2014-08","2014-09-16"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Teaching of English Sec Lang"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.A."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["English Placement Test (EPT)","Language Testing","Intelligilibility","Group oral format"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2014 Ji Eun Lee"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/50728"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["This study investigated raters’ judgments of test-takers’ intelligibility on Phase I of the EPT (English Placement Test), and how their various teaching experiences and different native languages affected their judgments. The EPT is administered to incoming international students whose TOEFL scores do not meet campus or departmental cutoff scores at the University of Illinois at Urbana Champaign. Phase I of the EPT oral section is designed to measure students’ intelligibility, and employs a group discussion format. Nine raters with different teaching experience and native languages were interviewed, and their responses were analyzed qualitatively. Results show that all the raters interpreted oral intelligibility differentially, and raters’ different understanding of the constructs underlying intelligibility resulted in different rating scales. This result wasn’t affected by raters’ teaching experience or language backgrounds. Rather than these, familiarity with a specific accent or rater’s individual ideas about intelligibility had more influence on their judgment. When they were asked about which specific features contribute more to intelligibility, most raters responded that segmental factors and lexical stress were important factors for their judgment. This might be because English Placement Test is a tool for assessing the ‘word level’ intelligibility. Phase I of the English Placement Test uses a one-on-one interview format or a group-administered format at raters’ discretion. Most raters reported that they prefer the one-on-one interview format. They argued that even though the group-administered format had the benefit of reducing time and expense, it would threaten the validity of the test because of the possibility of copying another test taker’s response or of getting nervous due to the situation in which test takers had to speak in front of group members. Providing a clarified and unified definition of intelligibility in the recalibration session is suggested. Additionally, raters still need to have sufficient knowledge of what the group oral discussion format is and know how to manage this format.","Item withdrawn by Laura Spradlin (lspradl2@illinois.edu) on 2014-05-08T13:15:38Z Item was in collections: University of Illinois Theses & Dissertations (ID: 1) No. of bitstreams: 2 Lee_Ji Eun.docx: 230358 bytes, checksum: f0c96f0cd0a60e27de6296e50acb1ad0 (MD5) Lee_Ji Eun.pdf: 950906 bytes, checksum: dba5fcc83f1b11a3dc1705fa1f464f66 (MD5)","Made available in DSpace on 2014-09-16T17:25:56Z (GMT). No. of bitstreams: 4 Ji Eun_Lee.pdf: 967404 bytes, checksum: 6f6b8663ce8fd8706380fe388c88060f (MD5) Lee_Ji Eun.docx: 275560 bytes, checksum: 78b79711c58dbca696acdac899bce86e (MD5) 1_Lee_Ji Eun.docx: 275560 bytes, checksum: 78b79711c58dbca696acdac899bce86e (MD5) license.txt: 4057 bytes, checksum: 0ec88c995b675e0b7ccd94a79189d716 (MD5)"]},{"key":"dc:title","label":"Title","values":["Raters' different judgments of intelligibility in the English placement test at University of Illinois at Urbana-Champaign"]}]}],"canonical_facts":{"dc:contributor":["Davidson, Frederick G."],"dc:creator":["Lee, Ji Eun"],"dc:date":["2014-09-16T17:25:56Z","2014-08","2014-09-16"],"dc:description":["This study investigated raters’ judgments of test-takers’ intelligibility on Phase I of the EPT (English Placement Test), and how their various teaching experiences and different native languages affected their judgments. The EPT is administered to incoming international students whose TOEFL scores do not meet campus or departmental cutoff scores at the University of Illinois at Urbana Champaign. Phase I of the EPT oral section is designed to measure students’ intelligibility, and employs a group discussion format. Nine raters with different teaching experience and native languages were interviewed, and their responses were analyzed qualitatively. Results show that all the raters interpreted oral intelligibility differentially, and raters’ different understanding of the constructs underlying intelligibility resulted in different rating scales. This result wasn’t affected by raters’ teaching experience or language backgrounds. Rather than these, familiarity with a specific accent or rater’s individual ideas about intelligibility had more influence on their judgment. When they were asked about which specific features contribute more to intelligibility, most raters responded that segmental factors and lexical stress were important factors for their judgment. This might be because English Placement Test is a tool for assessing the ‘word level’ intelligibility. Phase I of the English Placement Test uses a one-on-one interview format or a group-administered format at raters’ discretion. Most raters reported that they prefer the one-on-one interview format. They argued that even though the group-administered format had the benefit of reducing time and expense, it would threaten the validity of the test because of the possibility of copying another test taker’s response or of getting nervous due to the situation in which test takers had to speak in front of group members. Providing a clarified and unified definition of intelligibility in the recalibration session is suggested. Additionally, raters still need to have sufficient knowledge of what the group oral discussion format is and know how to manage this format.","Item withdrawn by Laura Spradlin (lspradl2@illinois.edu) on 2014-05-08T13:15:38Z Item was in collections: University of Illinois Theses & Dissertations (ID: 1) No. of bitstreams: 2 Lee_Ji Eun.docx: 230358 bytes, checksum: f0c96f0cd0a60e27de6296e50acb1ad0 (MD5) Lee_Ji Eun.pdf: 950906 bytes, checksum: dba5fcc83f1b11a3dc1705fa1f464f66 (MD5)","Made available in DSpace on 2014-09-16T17:25:56Z (GMT). No. of bitstreams: 4 Ji Eun_Lee.pdf: 967404 bytes, checksum: 6f6b8663ce8fd8706380fe388c88060f (MD5) Lee_Ji Eun.docx: 275560 bytes, checksum: 78b79711c58dbca696acdac899bce86e (MD5) 1_Lee_Ji Eun.docx: 275560 bytes, checksum: 78b79711c58dbca696acdac899bce86e (MD5) license.txt: 4057 bytes, checksum: 0ec88c995b675e0b7ccd94a79189d716 (MD5)"],"dc:identifier":["http://hdl.handle.net/2142/50728"],"dc:language":["en"],"dc:rights":["Copyright 2014 Ji Eun Lee"],"dc:subject":["English Placement Test (EPT)","Language Testing","Intelligilibility","Group oral format"],"dc:title":["Raters' different judgments of intelligibility in the English placement test at University of Illinois at Urbana-Champaign"],"dc:type":["text"],"thesis:degree_discipline":["Teaching of English Sec Lang"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.A."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:41Z"}