Analysing Student Code Submissions Using Program Analysis Techniques and Large Language Models

Programming students struggle to interpret technical diagnostics from static analysis tools, while large language models provide accessible explanations but lack formal guarantees. This thesis investigates when static analysis grounding—that is, providing the model with structured tool output (such as compiler warnings or type errors) to anchor its explanations—improves language model feedback across different programming contexts. Two experiments are reported: the first targeted concurrent programming and revealed deployment infrastructure as critical for evaluation; the second compared three feedback strategies for C programming—raw static analysis, analysis with fine-tuned model interpretation, and ungrounded language model—in a six-week classroom deployment across multiple assignments. Grounded feedback produces greater improvement for struggling students than either raw analysis or ungrounded models, with no benefit for competent programmers, suggesting grounding aids interpretation rather than detection: translating diagnostics, highlighting important issues, and connecting errors to concepts. Deployment infrastructure proved more consequential than technical optimisation, with cloud-native architecture enabling successful evaluation while sophisticated local deployment proved unreliable. The work contributes empirical evidence on when grounding benefits students, reusable static analysis infrastructure, and deployment design principles, with approximately 40–50% of infrastructure transferring across contexts and assignment characteristics significantly moderating feedback effectiveness.

Grantor dc:publisher

Stellenbosch : Stellenbosch University

Year dc:date.issued

2026

Author dc:creator

• Grobbelaar, Nelia

Advisors dc:contributor.advisor

• Inggs, Cornelia P.
• Bester, Willem H. K.

Language dc:language.iso

en

Repository record dc:identifier.uri

https://scholar.sun.ac.za/handle/10019.1/136044

OAI identifier oai:identifier

oai:scholar.sun.ac.za:10019.1/136044