{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/105126"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/105126","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Re-envisioning assessment of interlanguage pragmatic competence through computer-mediated communicative tasks","abstract":"Assessment of second language (L2) pragmatic knowledge is still a new and understudied area of research. Some researchers (Hudson, Detmer, & Brown, 1992, 1995; Roever, 2006, Walters, 2007) have played an important role in advancing the field, but their theories have followed speech act framework often criticized for pragmatic construct under-representation (Grabowski, 2009; Roever, 2011; Youn, 2015). Methodologically, past studies mainly used closed role-plays based on predetermined interactional outcomes (Youn, 2015). Kasper and Rose (2002) doubted the validity and authenticity of closed role-play tasks. In order to address these research gaps, and as a contribution to the general understanding of Second Language Pragmatic Testing (SLPT), this study combines second language pragmatics and computer-mediated communication to assess the pragmatic knowledge of second language users of English. It uses Purpura’s (2004) framework of communicative language ability for developing interactive, email-based role-play tasks to assess test takers’ pragmatic ability with regard to sociolinguistic, sociocultural and psychological meanings, and the use of polite formulaic expressions. Using mixed methods (Greene, 2007), qualitative and quantitative evidence was provided to support test inferences. A thorough needs analysis was conducted first through semi-structured interviews and then through an online survey by involving different ESL stakeholders including 153 faculty members at a large Mid-Western university. The online questionnaire results provided insight into the instructors’ perceptions of different email situations. The results of the needs analysis also helped to determine appropriate role-play situations. A set of communicative role-play tasks were developed following Davidson and Lynch’s (2002) test specification theory. Role-play cards were used to enhance standardization, and test takers were allowed to communicate naturally without following fixed interactional outcomes. A group of 52 graduate ESL students completed email role-play tasks. Two native-speaking raters evaluated the pragmatic ability of test takers and assigned scores using an empirically driven analytical scoring rubric on the email threads. Given that most students scored high, there appears to be a correlation between the ESL proficiency level of students and their scores in the pragmatic ability tests. Furthermore, inter-rater reliability analysis shows an overall high inter-rater reliability (0.85). There was some agreement between the hypothesized task difficulty typology and actual scores of three ability groups assessed in the present study. Qualitative analysis of interactive email data revealed a lack of knowledge of norms (as expected in Midwestern US academic settings) of appropriateness and politeness by the lower proficiency groups. Therefore, the low stakes test might have a great potential for developing instructional materials in an academic email communication context. Based on the findings of the present study, suggestions on inclusion of sociopragmatic competence into the ESL writing curriculum are made. Systematic curricular inclusion of email pragmatics in ESL courses will assist ESL learners in developing their email pragmatic competence in academic settings.","abstract_html":"Assessment of second language (L2) pragmatic knowledge is still a new and understudied area of research. Some researchers (Hudson, Detmer, & Brown, 1992, 1995; Roever, 2006, Walters, 2007) have played an important role in advancing the field, but their theories have followed speech act framework often criticized for pragmatic construct under-representation (Grabowski, 2009; Roever, 2011; Youn, 2015). Methodologically, past studies mainly used closed role-plays based on predetermined interactional outcomes (Youn, 2015). Kasper and Rose (2002) doubted the validity and authenticity of closed role-play tasks. In order to address these research gaps, and as a contribution to the general understanding of Second Language Pragmatic Testing (SLPT), this study combines second language pragmatics and computer-mediated communication to assess the pragmatic knowledge of second language users of English. It uses Purpura’s (2004) framework of communicative language ability for developing interactive, email-based role-play tasks to assess test takers’ pragmatic ability with regard to sociolinguistic, sociocultural and psychological meanings, and the use of polite formulaic expressions. Using mixed methods (Greene, 2007), qualitative and quantitative evidence was provided to support test inferences. A thorough needs analysis was conducted first through semi-structured interviews and then through an online survey by involving different ESL stakeholders including 153 faculty members at a large Mid-Western university. The online questionnaire results provided insight into the instructors’ perceptions of different email situations. The results of the needs analysis also helped to determine appropriate role-play situations. A set of communicative role-play tasks were developed following Davidson and Lynch’s (2002) test specification theory. Role-play cards were used to enhance standardization, and test takers were allowed to communicate naturally without following fixed interactional outcomes. A group of 52 graduate ESL students completed email role-play tasks. Two native-speaking raters evaluated the pragmatic ability of test takers and assigned scores using an empirically driven analytical scoring rubric on the email threads. Given that most students scored high, there appears to be a correlation between the ESL proficiency level of students and their scores in the pragmatic ability tests. Furthermore, inter-rater reliability analysis shows an overall high inter-rater reliability (0.85). There was some agreement between the hypothesized task difficulty typology and actual scores of three ability groups assessed in the present study. Qualitative analysis of interactive email data revealed a lack of knowledge of norms (as expected in Midwestern US academic settings) of appropriateness and politeness by the lower proficiency groups. Therefore, the low stakes test might have a great potential for developing instructional materials in an academic email communication context. Based on the findings of the present study, suggestions on inclusion of sociopragmatic competence into the ESL writing curriculum are made. Systematic curricular inclusion of email pragmatics in ESL courses will assist ESL learners in developing their email pragmatic competence in academic settings.","abstract_has_math":false,"creators":["Haider, Iftikhar"],"institution":"University of Illinois at Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Linguistics","degree_department":null,"school":null,"contributors":["Davidson, Fred","Bowles, Melissa","Sadler, Randall","Roever, Carsten"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2019,"date_issued":"2019-08-23T20:44:27Z","date_published":"2019-08-23T20:44:27Z","updated_at":"2026-07-22T22:24:44Z","subjects":["Language Assessment","Interlanguage Pragmatics","Computer-Mediated","Interactive","Testing","Communicative","ESL"],"languages":["en"],"rights":["Copyright 2019 Iftikhar Haider"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/105126","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Davidson, Fred","Bowles, Melissa","Sadler, Randall","Roever, Carsten"]},{"key":"dc:creator","label":"Author","values":["Haider, Iftikhar"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2019-08-23T20:44:27Z","2021-08-24T09:15:20Z","2019-04-19","2019-05"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Linguistics"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Language Assessment","Interlanguage Pragmatics","Computer-Mediated","Interactive","Testing","Communicative","ESL"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2019 Iftikhar Haider"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/105126"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Assessment of second language (L2) pragmatic knowledge is still a new and understudied area of research. Some researchers (Hudson, Detmer, & Brown, 1992, 1995; Roever, 2006, Walters, 2007) have played an important role in advancing the field, but their theories have followed speech act framework often criticized for pragmatic construct under-representation (Grabowski, 2009; Roever, 2011; Youn, 2015). Methodologically, past studies mainly used closed role-plays based on predetermined interactional outcomes (Youn, 2015). Kasper and Rose (2002) doubted the validity and authenticity of closed role-play tasks. In order to address these research gaps, and as a contribution to the general understanding of Second Language Pragmatic Testing (SLPT), this study combines second language pragmatics and computer-mediated communication to assess the pragmatic knowledge of second language users of English. It uses Purpura’s (2004) framework of communicative language ability for developing interactive, email-based role-play tasks to assess test takers’ pragmatic ability with regard to sociolinguistic, sociocultural and psychological meanings, and the use of polite formulaic expressions. Using mixed methods (Greene, 2007), qualitative and quantitative evidence was provided to support test inferences. A thorough needs analysis was conducted first through semi-structured interviews and then through an online survey by involving different ESL stakeholders including 153 faculty members at a large Mid-Western university. The online questionnaire results provided insight into the instructors’ perceptions of different email situations. The results of the needs analysis also helped to determine appropriate role-play situations. A set of communicative role-play tasks were developed following Davidson and Lynch’s (2002) test specification theory. Role-play cards were used to enhance standardization, and test takers were allowed to communicate naturally without following fixed interactional outcomes. A group of 52 graduate ESL students completed email role-play tasks. Two native-speaking raters evaluated the pragmatic ability of test takers and assigned scores using an empirically driven analytical scoring rubric on the email threads. Given that most students scored high, there appears to be a correlation between the ESL proficiency level of students and their scores in the pragmatic ability tests. Furthermore, inter-rater reliability analysis shows an overall high inter-rater reliability (0.85). There was some agreement between the hypothesized task difficulty typology and actual scores of three ability groups assessed in the present study. Qualitative analysis of interactive email data revealed a lack of knowledge of norms (as expected in Midwestern US academic settings) of appropriateness and politeness by the lower proficiency groups. Therefore, the low stakes test might have a great potential for developing instructional materials in an academic email communication context. Based on the findings of the present study, suggestions on inclusion of sociopragmatic competence into the ESL writing curriculum are made. Systematic curricular inclusion of email pragmatics in ESL courses will assist ESL learners in developing their email pragmatic competence in academic settings.","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2021-05-01","The student, Iftikhar Haider, accepted the attached license on 2019-01-12 at 12:47.","The student, Iftikhar Haider, submitted this Dissertation for approval on 2019-01-12 at 13:20.","This Dissertation was approved for publication on 2019-04-19 at 07:56.","DSpace SAF Submission Ingestion Package generated from Vireo submission #13355 on 2019-08-22 at 16:19:42","Made available in DSpace on 2019-08-23T20:44:27Z (GMT). No. of bitstreams: 3 HAIDER-DISSERTATION-2019.pdf: 1789574 bytes, checksum: afde9c112561852d1a89a435f907e371 (MD5) LICENSE.txt: 4212 bytes, checksum: b412d9fa16f3824b5d3b4f8da3379341 (MD5) PROQUEST_LICENSE.txt: 4558 bytes, checksum: 4daf856737e653163fb39d848eb6a9ac (MD5) Previous issue date: 2019-04-19","Embargo set by: Seth Robbins for item 112245 Lift date: 2021-08-23T20:44:50Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 112245 Lift date: 2021-08-23T20:46:41Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 112245 Lift date: 2021-08-23T20:47:38Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 112245 Lift date: 2021-08-23T20:48:32Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Limited Restriction Lifted for Item 112245 on 2021-08-24T09:15:20Z."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Re-envisioning assessment of interlanguage pragmatic competence through computer-mediated communicative tasks"]}]}],"canonical_facts":{"dc:contributor":["Davidson, Fred","Bowles, Melissa","Sadler, Randall","Roever, Carsten"],"dc:creator":["Haider, Iftikhar"],"dc:date":["2019-08-23T20:44:27Z","2021-08-24T09:15:20Z","2019-04-19","2019-05"],"dc:description":["Assessment of second language (L2) pragmatic knowledge is still a new and understudied area of research. Some researchers (Hudson, Detmer, & Brown, 1992, 1995; Roever, 2006, Walters, 2007) have played an important role in advancing the field, but their theories have followed speech act framework often criticized for pragmatic construct under-representation (Grabowski, 2009; Roever, 2011; Youn, 2015). Methodologically, past studies mainly used closed role-plays based on predetermined interactional outcomes (Youn, 2015). Kasper and Rose (2002) doubted the validity and authenticity of closed role-play tasks. In order to address these research gaps, and as a contribution to the general understanding of Second Language Pragmatic Testing (SLPT), this study combines second language pragmatics and computer-mediated communication to assess the pragmatic knowledge of second language users of English. It uses Purpura’s (2004) framework of communicative language ability for developing interactive, email-based role-play tasks to assess test takers’ pragmatic ability with regard to sociolinguistic, sociocultural and psychological meanings, and the use of polite formulaic expressions. Using mixed methods (Greene, 2007), qualitative and quantitative evidence was provided to support test inferences. A thorough needs analysis was conducted first through semi-structured interviews and then through an online survey by involving different ESL stakeholders including 153 faculty members at a large Mid-Western university. The online questionnaire results provided insight into the instructors’ perceptions of different email situations. The results of the needs analysis also helped to determine appropriate role-play situations. A set of communicative role-play tasks were developed following Davidson and Lynch’s (2002) test specification theory. Role-play cards were used to enhance standardization, and test takers were allowed to communicate naturally without following fixed interactional outcomes. A group of 52 graduate ESL students completed email role-play tasks. Two native-speaking raters evaluated the pragmatic ability of test takers and assigned scores using an empirically driven analytical scoring rubric on the email threads. Given that most students scored high, there appears to be a correlation between the ESL proficiency level of students and their scores in the pragmatic ability tests. Furthermore, inter-rater reliability analysis shows an overall high inter-rater reliability (0.85). There was some agreement between the hypothesized task difficulty typology and actual scores of three ability groups assessed in the present study. Qualitative analysis of interactive email data revealed a lack of knowledge of norms (as expected in Midwestern US academic settings) of appropriateness and politeness by the lower proficiency groups. Therefore, the low stakes test might have a great potential for developing instructional materials in an academic email communication context. Based on the findings of the present study, suggestions on inclusion of sociopragmatic competence into the ESL writing curriculum are made. Systematic curricular inclusion of email pragmatics in ESL courses will assist ESL learners in developing their email pragmatic competence in academic settings.","Submission published under a 24 month embargo labeled 'Closed Access', the embargo will last until 2021-05-01","The student, Iftikhar Haider, accepted the attached license on 2019-01-12 at 12:47.","The student, Iftikhar Haider, submitted this Dissertation for approval on 2019-01-12 at 13:20.","This Dissertation was approved for publication on 2019-04-19 at 07:56.","DSpace SAF Submission Ingestion Package generated from Vireo submission #13355 on 2019-08-22 at 16:19:42","Made available in DSpace on 2019-08-23T20:44:27Z (GMT). No. of bitstreams: 3 HAIDER-DISSERTATION-2019.pdf: 1789574 bytes, checksum: afde9c112561852d1a89a435f907e371 (MD5) LICENSE.txt: 4212 bytes, checksum: b412d9fa16f3824b5d3b4f8da3379341 (MD5) PROQUEST_LICENSE.txt: 4558 bytes, checksum: 4daf856737e653163fb39d848eb6a9ac (MD5) Previous issue date: 2019-04-19","Embargo set by: Seth Robbins for item 112245 Lift date: 2021-08-23T20:44:50Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 112245 Lift date: 2021-08-23T20:46:41Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 112245 Lift date: 2021-08-23T20:47:38Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Embargo set by: Seth Robbins for item 112245 Lift date: 2021-08-23T20:48:32Z Reason: Author requested closed access (OA after 2yrs) in Vireo ETD system","Limited Restriction Lifted for Item 112245 on 2021-08-24T09:15:20Z."],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/105126"],"dc:language":["en"],"dc:rights":["Copyright 2019 Iftikhar Haider"],"dc:subject":["Language Assessment","Interlanguage Pragmatics","Computer-Mediated","Interactive","Testing","Communicative","ESL"],"dc:title":["Re-envisioning assessment of interlanguage pragmatic competence through computer-mediated communicative tasks"],"dc:type":["text"],"thesis:degree_discipline":["Linguistics"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:44Z"}