{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/125529"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/125529","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"What makes an argument effective? Triangulating task characteristics, essay performances, and rating processes in an L2 integrated argumentative writing test","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-02-04 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-02-04 without embargo terms","abstract_has_math":false,"creators":["Chuang, Ping-Lin"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Linguistics","degree_department":null,"school":null,"contributors":["Yan, Xun","Bowles, Melissa","Christianson, Kiel","Plakans, Lia"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-08","date_published":"2024-08","updated_at":"2026-07-22T22:25:02Z","subjects":["L2 Writing Assessment","Integrated Writing","Argumentation","Source Use","Rater Behavior"],"languages":["en","eng"],"rights":["Copyright 2024 Ping-Lin Chuang"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/125529","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Yan, Xun","Bowles, Melissa","Christianson, Kiel","Plakans, Lia"]},{"key":"dc:creator","label":"Author","values":["Chuang, Ping-Lin"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2024-08","2024-06-27"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Linguistics"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["L2 Writing Assessment","Integrated Writing","Argumentation","Source Use","Rater Behavior"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2024 Ping-Lin Chuang"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/125529"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-02-04 without embargo terms","The student, Ping-Lin Chuang, accepted the attached license on 2024-06-21 at 13:35.","The student, Ping-Lin Chuang, submitted this Dissertation for approval on 2024-06-21 at 13:45.","This Dissertation was approved for publication on 2024-06-27 at 08:56.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20857 on 2025-02-04 at 21:03:47","Integrated argumentative tasks are commonly used to assess learners’ writing ability in second language (L2) writing assessment. While abundant research has addressed performance characteristics and rating processes of integrated argumentative tests (e.g., Gebril & Plakans, 2009, 2014), our understanding of the nature of L2 argumentation and the impact of source use on argumentation quality remains relatively limited. This dissertation study thus investigates essay performances and rating processes of an operational integrated argumentative writing test to delineate how features related to task, examinee, and rater interact, in the hope of gathering validity evidence for the use of such tests in an L2 context. Specifically, it examines (1) whether argumentation effectiveness exhibits systematic differences across proficiency levels; (2) how source use influences argumentation effectiveness; (3) whether raters consider source use characteristics when scoring argumentation effectiveness; (4) what cognitive processes are involved in scoring argumentation effectiveness. To answer the four main research questions, I analyzed test takers’ written products and raters’ rating performances qualitatively and quantitatively. I first conducted text analysis to examine features pertaining to argumentation and source use. Then, I designed an offline experiment to examine if different source use characteristics impact raters’ scoring of argumentation. I further investigated raters’ cognitive processes using the eye-tracking methodology. Results from test takers’ writing performances show that (1) both argument structure and reasoning quality differed across proficiency levels and (2) source use characteristics had an impact on argument structure and reasoning quality. The offline experiment further indicates that the quantity and quality of source use could determine argumentation scores assigned by raters. Finally, the eye-tracking experiment reveals the cognitive processes involved in scoring integrated argumentative writing performances. Raters paid the most attention to sentences with source text use, and their eye movements were related to the scores and comments they provided. This dissertation provides a deeper understanding of how L2 test takers construct arguments and how raters evaluate integrated argumentative writing performances. It also stands to have practical implications for L2 writing pedagogy and assessment, including how argumentation can be taught in writing courses and how rating scales and rater training programs can be developed to guide the scoring of argumentation effectiveness in academic contexts."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["What makes an argument effective? Triangulating task characteristics, essay performances, and rating processes in an L2 integrated argumentative writing test"]}]}],"canonical_facts":{"dc:contributor":["Yan, Xun","Bowles, Melissa","Christianson, Kiel","Plakans, Lia"],"dc:creator":["Chuang, Ping-Lin"],"dc:date":["2024-08","2024-06-27"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-02-04 without embargo terms","The student, Ping-Lin Chuang, accepted the attached license on 2024-06-21 at 13:35.","The student, Ping-Lin Chuang, submitted this Dissertation for approval on 2024-06-21 at 13:45.","This Dissertation was approved for publication on 2024-06-27 at 08:56.","DSpace SAF Submission Ingestion Package generated from Vireo submission #20857 on 2025-02-04 at 21:03:47","Integrated argumentative tasks are commonly used to assess learners’ writing ability in second language (L2) writing assessment. While abundant research has addressed performance characteristics and rating processes of integrated argumentative tests (e.g., Gebril & Plakans, 2009, 2014), our understanding of the nature of L2 argumentation and the impact of source use on argumentation quality remains relatively limited. This dissertation study thus investigates essay performances and rating processes of an operational integrated argumentative writing test to delineate how features related to task, examinee, and rater interact, in the hope of gathering validity evidence for the use of such tests in an L2 context. Specifically, it examines (1) whether argumentation effectiveness exhibits systematic differences across proficiency levels; (2) how source use influences argumentation effectiveness; (3) whether raters consider source use characteristics when scoring argumentation effectiveness; (4) what cognitive processes are involved in scoring argumentation effectiveness. To answer the four main research questions, I analyzed test takers’ written products and raters’ rating performances qualitatively and quantitatively. I first conducted text analysis to examine features pertaining to argumentation and source use. Then, I designed an offline experiment to examine if different source use characteristics impact raters’ scoring of argumentation. I further investigated raters’ cognitive processes using the eye-tracking methodology. Results from test takers’ writing performances show that (1) both argument structure and reasoning quality differed across proficiency levels and (2) source use characteristics had an impact on argument structure and reasoning quality. The offline experiment further indicates that the quantity and quality of source use could determine argumentation scores assigned by raters. Finally, the eye-tracking experiment reveals the cognitive processes involved in scoring integrated argumentative writing performances. Raters paid the most attention to sentences with source text use, and their eye movements were related to the scores and comments they provided. This dissertation provides a deeper understanding of how L2 test takers construct arguments and how raters evaluate integrated argumentative writing performances. It also stands to have practical implications for L2 writing pedagogy and assessment, including how argumentation can be taught in writing courses and how rating scales and rater training programs can be developed to guide the scoring of argumentation effectiveness in academic contexts."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/125529"],"dc:language":["en","eng"],"dc:rights":["Copyright 2024 Ping-Lin Chuang"],"dc:subject":["L2 Writing Assessment","Integrated Writing","Argumentation","Source Use","Rater Behavior"],"dc:title":["What makes an argument effective? Triangulating task characteristics, essay performances, and rating processes in an L2 integrated argumentative writing test"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Linguistics"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:02Z"}