{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/105071"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/105071","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"A multidimensional approach for analyzing variants of code writing questions in a CS1 course","abstract":"To defend against collaborative cheating in code writing questions, instructors of CS1 courses with asynchronous exams can use the strategy of question variants, being manually written questions to be selected at random to assess the same learning goal. In order to create these variants, currently the instructors have to rely on intuition to accomplish the competing goals of ensuring variants are different enough to defend against collaborative cheating, and yet similar enough where students are assessed fairly. In this paper, we propose a multidimensional approach of analyzing these variants. We apply our approach on a dataset of 3 midterm exams from a large CS1 course. Our results show that (1) observable inequalities exist between variants and (2) these differences are not just limited to score. Our results also show that the information gathered from our analysis approach can be used to provide recommendations for improving design of future variants.","abstract_html":"To defend against collaborative cheating in code writing questions, instructors of CS1 courses with asynchronous exams can use the strategy of question variants, being manually written questions to be selected at random to assess the same learning goal. In order to create these variants, currently the instructors have to rely on intuition to accomplish the competing goals of ensuring variants are different enough to defend against collaborative cheating, and yet similar enough where students are assessed fairly. In this paper, we propose a multidimensional approach of analyzing these variants. We apply our approach on a dataset of 3 midterm exams from a large CS1 course. Our results show that (1) observable inequalities exist between variants and (2) these differences are not just limited to score. Our results also show that the information gathered from our analysis approach can be used to provide recommendations for improving design of future variants.","abstract_has_math":false,"creators":["Butler, Liia M."],"institution":"University of Illinois at Urbana-Champaign","degree_name":"M.S.","degree_level":"Thesis","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Xie, Tao","Challen, Geoffrey"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2019,"date_issued":"2019-08-23T20:36:05Z","date_published":"2019-08-23T20:36:05Z","updated_at":"2026-07-22T22:24:44Z","subjects":["Question Variants","CS1","code writing","exam questions","assessment"],"languages":["en"],"rights":["Copyright 2019 Liia Butler"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"http://hdl.handle.net/2142/105071","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Xie, Tao","Challen, Geoffrey"]},{"key":"dc:creator","label":"Author","values":["Butler, Liia M."]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2019-08-23T20:36:05Z","2021-08-24T09:15:20Z","2019-04-22","2019-05"]},{"key":"dc:type","label":"Dc Type","values":["text"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois at Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Question Variants","CS1","code writing","exam questions","assessment"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2019 Liia Butler"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["http://hdl.handle.net/2142/105071"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["To defend against collaborative cheating in code writing questions, instructors of CS1 courses with asynchronous exams can use the strategy of question variants, being manually written questions to be selected at random to assess the same learning goal. In order to create these variants, currently the instructors have to rely on intuition to accomplish the competing goals of ensuring variants are different enough to defend against collaborative cheating, and yet similar enough where students are assessed fairly. In this paper, we propose a multidimensional approach of analyzing these variants. We apply our approach on a dataset of 3 midterm exams from a large CS1 course. Our results show that (1) observable inequalities exist between variants and (2) these differences are not just limited to score. Our results also show that the information gathered from our analysis approach can be used to provide recommendations for improving design of future variants.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2021-05-01","The student, Liia Butler, accepted the attached license on 2019-04-22 at 10:22.","The student, Liia Butler, submitted this Thesis for approval on 2019-04-22 at 13:13.","This Thesis was approved for publication on 2019-04-22 at 13:57.","DSpace SAF Submission Ingestion Package generated from Vireo submission #13814 on 2019-08-22 at 15:07:39","Made available in DSpace on 2019-08-23T20:36:05Z (GMT). No. of bitstreams: 2 BUTLER-THESIS-2019.pdf: 473769 bytes, checksum: ce13c800b985e0f0d35d00de708e4e4f (MD5) LICENSE.txt: 4208 bytes, checksum: 10e662264d2a58ec18577562c2dc824e (MD5) Previous issue date: 2019-04-22","Embargo set by: Seth Robbins for item 112190 Lift date: 2021-08-23T20:36:18Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only Restriction Lifted for Item 112190 on 2021-08-24T09:15:20Z."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["A multidimensional approach for analyzing variants of code writing questions in a CS1 course"]}]}],"canonical_facts":{"dc:contributor":["Xie, Tao","Challen, Geoffrey"],"dc:creator":["Butler, Liia M."],"dc:date":["2019-08-23T20:36:05Z","2021-08-24T09:15:20Z","2019-04-22","2019-05"],"dc:description":["To defend against collaborative cheating in code writing questions, instructors of CS1 courses with asynchronous exams can use the strategy of question variants, being manually written questions to be selected at random to assess the same learning goal. In order to create these variants, currently the instructors have to rely on intuition to accomplish the competing goals of ensuring variants are different enough to defend against collaborative cheating, and yet similar enough where students are assessed fairly. In this paper, we propose a multidimensional approach of analyzing these variants. We apply our approach on a dataset of 3 midterm exams from a large CS1 course. Our results show that (1) observable inequalities exist between variants and (2) these differences are not just limited to score. Our results also show that the information gathered from our analysis approach can be used to provide recommendations for improving design of future variants.","Submission published under a 24 month embargo labeled 'U of I Access', the embargo will last until 2021-05-01","The student, Liia Butler, accepted the attached license on 2019-04-22 at 10:22.","The student, Liia Butler, submitted this Thesis for approval on 2019-04-22 at 13:13.","This Thesis was approved for publication on 2019-04-22 at 13:57.","DSpace SAF Submission Ingestion Package generated from Vireo submission #13814 on 2019-08-22 at 15:07:39","Made available in DSpace on 2019-08-23T20:36:05Z (GMT). No. of bitstreams: 2 BUTLER-THESIS-2019.pdf: 473769 bytes, checksum: ce13c800b985e0f0d35d00de708e4e4f (MD5) LICENSE.txt: 4208 bytes, checksum: 10e662264d2a58ec18577562c2dc824e (MD5) Previous issue date: 2019-04-22","Embargo set by: Seth Robbins for item 112190 Lift date: 2021-08-23T20:36:18Z Reason: Author requested U of Illinois access only (OA after 2yrs) in Vireo ETD system","U of I Only Restriction Lifted for Item 112190 on 2021-08-24T09:15:20Z."],"dc:format":["application/pdf"],"dc:identifier":["http://hdl.handle.net/2142/105071"],"dc:language":["en"],"dc:rights":["Copyright 2019 Liia Butler"],"dc:subject":["Question Variants","CS1","code writing","exam questions","assessment"],"dc:title":["A multidimensional approach for analyzing variants of code writing questions in a CS1 course"],"dc:type":["text"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S."],"thesis:institution_name":["University of Illinois at Urbana-Champaign"]},"updated_at":"2026-07-22T22:24:44Z"}