Explore the moving sample size used in a procedure for detecting compromised items based on item response theory

A computerized adaptive testing (CAT) system can provide tests for examinees continuously, so items in the item bank face the risk of being leaked. The IRT-based detection procedure developed by Zhang and Li (2016) is further studied in this thesis, which is designed to identify compromised items via a series of hypothesis tests. The moving sample size m in the Zhang and Li procedure is very important because it can affect the lag and the power of the procedure in identifying compromised items. In this thesis, the author will explore the relationship between the moving sample size and the lag, and the relationship between the moving sample size and the power. The simulation results indicate that at the significance level of 0.05, moving sample size m has an exponential relationship with the lag, and has a quadratic relationship with the power. At the significance level of 0.01, moving sample size m has a logarithmic relationship with the lag, and has a cubic relationship with the power. The author attempts to find an appropriate value of moving sample size m to achieve a large value of the power and a small value of the lag. The simulation results show that at both significance levels 0.05 and 0.01, setting the moving sample size m to 200 can accomplish the mission, while setting moving sample size m to 10 will decrease the performance of the procedure in the case that the significance level is 0.05.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Educational Psychology

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2018

Author dc:creator

• Jie, Yongjing

Contributors dc:contributor

• Zhang, Jinming
• Chang, Hua-Hua
• Anderson, Carolyn J.

• Item Response Theory (IRT)-based detection procedure
• Moving sample size
• Power
• Lag

Statement dc:rights

• Copyright 2017 Yongjing Jie

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/99519

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/99519