{"id":{"repo_id":"uiuc","oai_identifier":"oai:www.ideals.illinois.edu:2142/129414"},"canonical_url":"https://search.dev.ndltd.org/etd/uiuc/oai:www.ideals.illinois.edu:2142/129414","repository":{"repo_id":"uiuc","name":"University of Illinois - Urbana-Champaign","base_url":"https://www.ideals.illinois.edu/oai-pmh"},"display":{"title":"Discovering, autogenerating, and evaluating distractors for Parsons problems in CS1","abstract":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-19 without embargo terms","abstract_html":"Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-19 without embargo terms","abstract_has_math":false,"creators":["Smith IV, David Hamilton"],"institution":"University of Illinois Urbana-Champaign","degree_name":"Ph.D.","degree_level":"Dissertation","degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Zilles, Craig","Lewis, Colleen","Ericson, Barbara","Cunningham, Kathryn"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2025,"date_issued":"2025-04-23","date_published":"2025-04-23","updated_at":"2026-07-22T22:25:05Z","subjects":["Computing Education","Parsons problems","CS1","Distractors","Item Response Theory","Classical Test Theory","Measurement Theory"],"languages":["en","eng"],"rights":["Copyright 2025 David Smith IV"],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/2142/129414","outbound_label":"Handle","outbound_source":"dc:identifier"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Zilles, Craig","Lewis, Colleen","Ericson, Barbara","Cunningham, Kathryn"]},{"key":"dc:creator","label":"Author","values":["Smith IV, David Hamilton"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2025-04-23","2025-05"]},{"key":"dc:type","label":"Dc Type","values":["text","Thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Dissertation"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D."]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of Illinois Urbana-Champaign"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Computing Education","Parsons problems","CS1","Distractors","Item Response Theory","Classical Test Theory","Measurement Theory"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language","label":"Dc Language","values":["en","eng"]},{"key":"dc:rights","label":"Dc Rights","values":["Copyright 2025 David Smith IV"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://hdl.handle.net/2142/129414"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-19 without embargo terms","The student, David Smith IV, accepted the attached license on 2025-04-20 at 23:12.","The student, David Smith IV, submitted this Dissertation for approval on 2025-04-20 at 23:22.","This Dissertation was approved for publication on 2025-04-23 at 14:43.","DSpace SAF Submission Ingestion Package generated from Vireo submission #21851 on 2025-10-19 at 18:18:27","This dissertation presents work on evaluating the use of distractor blocks in Parsons problems, a problem type aimed at supporting novice programmers in learning to write code. Since their introduction by Parsons & Hayden [1], distractors have become common place in studies investigating Parsons problems. However, prior to the work presented in this dissertation, there had been a limited number of studies investigating their utility. In filling this gap, my work has focused on investigating methods of developing distractors and the impact of distractors in both summative and formative contexts. I first introduce a method of creating distractor templates from analysis of code writing errors and using those templates to support the autogeneration of distractors [2], a process that has been integrated into CodeSpec, a programming practice and eTextbook platform [3]. I present a series of studies comparing Parsons problems on exams that include distractors to those that do not. These studies have also included a comparison between different methods of including distractors, specifically comparing the use of distractors that are jumbled among the other options to distractors that are visually-grouped with their correct alternative [4, 5]. Additionally, I present a study comparing the impact on learning gains when practicing with Parsons problems that include distractors versus those without distractors. Learning gains are measured through pre- and post-assessment performance. In support of this work, I have also added the ability to associate feedback with distractors in the PrairieLearn assessment platform. The findings presented suggest that in summative contexts the presence of a single distractor group does not meaningfully increase question difficulty or item quality while causing students to spend significantly more time on the problems, a concern in a time constrained context such as an exam. However, as more distractor groups are added difficulty continues to increase with minimal impact—either positively or negatively—to item quality. This suggests that when distractors are added in sufficient quantities they can be useful for reducing the ceiling effect on score we observed in Parsons problems used in this study that do not include distractors. In formative contexts, the findings I present suggest that the presence of distractors causes students to engage more deeply with the code presented in blocks they are grouped with. The results of a randomized controlled trial indicate that students practicing with questions including distractors had measurable learning gains as measured through the post-test."]},{"key":"dc:format","label":"Dc Format","values":["application/pdf"]},{"key":"dc:title","label":"Title","values":["Discovering, autogenerating, and evaluating distractors for Parsons problems in CS1"]}]}],"canonical_facts":{"dc:contributor":["Zilles, Craig","Lewis, Colleen","Ericson, Barbara","Cunningham, Kathryn"],"dc:creator":["Smith IV, David Hamilton"],"dc:date":["2025-04-23","2025-05"],"dc:description":["Submission original under an indefinite embargo labeled 'Open Access'. The submission was exported from vireo on 2025-10-19 without embargo terms","The student, David Smith IV, accepted the attached license on 2025-04-20 at 23:12.","The student, David Smith IV, submitted this Dissertation for approval on 2025-04-20 at 23:22.","This Dissertation was approved for publication on 2025-04-23 at 14:43.","DSpace SAF Submission Ingestion Package generated from Vireo submission #21851 on 2025-10-19 at 18:18:27","This dissertation presents work on evaluating the use of distractor blocks in Parsons problems, a problem type aimed at supporting novice programmers in learning to write code. Since their introduction by Parsons & Hayden [1], distractors have become common place in studies investigating Parsons problems. However, prior to the work presented in this dissertation, there had been a limited number of studies investigating their utility. In filling this gap, my work has focused on investigating methods of developing distractors and the impact of distractors in both summative and formative contexts. I first introduce a method of creating distractor templates from analysis of code writing errors and using those templates to support the autogeneration of distractors [2], a process that has been integrated into CodeSpec, a programming practice and eTextbook platform [3]. I present a series of studies comparing Parsons problems on exams that include distractors to those that do not. These studies have also included a comparison between different methods of including distractors, specifically comparing the use of distractors that are jumbled among the other options to distractors that are visually-grouped with their correct alternative [4, 5]. Additionally, I present a study comparing the impact on learning gains when practicing with Parsons problems that include distractors versus those without distractors. Learning gains are measured through pre- and post-assessment performance. In support of this work, I have also added the ability to associate feedback with distractors in the PrairieLearn assessment platform. The findings presented suggest that in summative contexts the presence of a single distractor group does not meaningfully increase question difficulty or item quality while causing students to spend significantly more time on the problems, a concern in a time constrained context such as an exam. However, as more distractor groups are added difficulty continues to increase with minimal impact—either positively or negatively—to item quality. This suggests that when distractors are added in sufficient quantities they can be useful for reducing the ceiling effect on score we observed in Parsons problems used in this study that do not include distractors. In formative contexts, the findings I present suggest that the presence of distractors causes students to engage more deeply with the code presented in blocks they are grouped with. The results of a randomized controlled trial indicate that students practicing with questions including distractors had measurable learning gains as measured through the post-test."],"dc:format":["application/pdf"],"dc:identifier":["https://hdl.handle.net/2142/129414"],"dc:language":["en","eng"],"dc:rights":["Copyright 2025 David Smith IV"],"dc:subject":["Computing Education","Parsons problems","CS1","Distractors","Item Response Theory","Classical Test Theory","Measurement Theory"],"dc:title":["Discovering, autogenerating, and evaluating distractors for Parsons problems in CS1"],"dc:type":["text","Thesis"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_level":["Dissertation"],"thesis:degree_name":["Ph.D."],"thesis:institution_name":["University of Illinois Urbana-Champaign"]},"updated_at":"2026-07-22T22:25:05Z"}