Developing and evaluating domain models for programming skills using learning curve analysis

Identifying key concepts in programming is important for accurately tracking skill development and designing better support mechanisms for students. Prior research has identified a plethora of skills at varying levels of granularity, from broad abilities such as code comprehension and tracing to fine-grained skills like using individual syntactic elements correctly. However, more evidence is required to understand the extent these skill models reflect the actual skill acquisition process of students. Knowledge components, which are acquired units of skills and abilities inferred from the performance on a set of tasks, can be an appropriate starting point in a framework for evaluating these skill models and generating new models in a data-driven manner. These knowledge components can be evaluated by learning curve analysis, which is an educational data mining technique for modeling skill development using data on problem-solving performance of students. Yet, previous applications of learning curve analysis for programming could not identify a robust and interpretable skill model, which may imply that programming skills are more complex than initially assumed. In this thesis, we summarize two studies where we evaluate two domain models that explain student skill development. The first study is a replication of prior work that proposed the use of syntactic structures in a programming language as individual skills in programming. The second study proposes a novel domain model that uses programming plans from computing education literature to model student skills. We evaluate the extent to which these domain models can explain students' development across homework assignments in an introductory programming course using data collected from seven semesters. Our findings imply that learning curve analysis can be used to produce useful insights from solutions to open-ended code-writing exercises.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois Urbana-Champaign

Year dc:date

2025

Author dc:creator

• Demirtas, Mehmet Arif

Contributors dc:contributor

• Cunningham, Kathryn I

• computing education
• educational data mining

Statement dc:rights

• Copyright 2025 Mehmet Arif Demirtas

Language dc:language

en

Handle dc:identifier

https://hdl.handle.net/2142/132550

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/132550