A multidimensional approach for analyzing variants of code writing questions in a CS1 course

To defend against collaborative cheating in code writing questions, instructors of CS1 courses with asynchronous exams can use the strategy of question variants, being manually written questions to be selected at random to assess the same learning goal. In order to create these variants, currently the instructors have to rely on intuition to accomplish the competing goals of ensuring variants are different enough to defend against collaborative cheating, and yet similar enough where students are assessed fairly. In this paper, we propose a multidimensional approach of analyzing these variants. We apply our approach on a dataset of 3 midterm exams from a large CS1 course. Our results show that (1) observable inequalities exist between variants and (2) these differences are not just limited to score. Our results also show that the information gathered from our analysis approach can be used to provide recommendations for improving design of future variants.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2019

Author dc:creator

• Butler, Liia M.

Contributors dc:contributor

• Xie, Tao
• Challen, Geoffrey

• Question Variants
• CS1
• code writing
• exam questions
• assessment

Statement dc:rights

• Copyright 2019 Liia Butler

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/105071

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/105071