Towards more effective testing and debugging education in computer science courses

Testing and debugging are essential skills for ensuring code correctness in programming, yet are not often emphasized in computer science instruction. Further, while autograders can provide feedback to students quickly, students can become reliant on the autograder as the primary means for determining correctness of their code, rather than developing their own testing and debugging skills. In this dissertation, I first present a policy to help promote only correct coding submissions to the autograder. We were able to see a modest increase in lab success measures with respect to our goals. I next present a debugging exercise inspired by the pedagogical technique, Interactive Lecture Demonstrations (ILDs). During ILDs, students predict a demonstration's result, experience the demonstration, and then reflect on their experience. I adapted this process for debugging (ILDBug) by having students look at example program outputs and predict the bug, then they experience debugging by tracing the code and trying to fix the bug, and finally students reflect on their debugging process. We engaged students in a series of three ILDBug exercises during a lab section of an introductory programming course for non-CS, engineering majors. To evaluate whether my exercises were improving students’ debugging skills, I introduced a cross-over study design with three populations: each group of students completed the same three exercises but in a different order. I then compared how students performed on each exercise when it was their first exercise or their last exercise. We graded students' predictions for accuracy using a binary score and measured how long students took to fix the bugs using click-stream logs. Students generally fixed bugs faster after completing two ILDBug exercises. Students improved at predicting bugs that were moderately difficult to identify. With the promise of ILDBug after our initial study, our final study evaluates the effectiveness of ILDBug. I created two different versions of a debugging lab using the ILDBug exercise design. One focused more on hard-coding bugs, the other focused on iteration bugs. I then present two debugging questions in a future quiz, one hard-coding bug and one loop bug. I tested to see if students from one lab performed better on their respective quiz bug than students who had the other lab for both questions. There were no significant differences in performance between students who had one lab version versus another. I discuss my findings and how they align with prior literature. I also discuss future research directions of the role ILDBug can play in explicit debugging instruction.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Butler, Liia

Contributors dc:contributor

• Herman, Geoffrey L
• Lewis, Colleen M
• Zilles, Craig
• Heckman, Sarah

• debugging
• computing education
• programming education

Statement dc:rights

• Copyright 2025 Liia Butler

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/129210

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F129210