Item parameter drift and online calibration

An important assumption of item response theory based computerized adaptive assessment is item parameter invariance. Sometimes, however, item parameters are not invariant across different test administrations due to factors other than sampling error; and this phenomenon is termed item parameter drift. Several methods have been developed to detect drifted items, and most of the them were designed to detect drifts in the unidimensional item response model under the paper and pencil testing framework, which may not be adequate for computerized adaptive testing. This paper introduces an online (re)calibration design to detect item parameter drift for computerized adaptive testings in both unidimensional and multidimensional environment. Specifically, for online calibra- tion optimal design in unidimensional computerized adaptive testing model, a modified two-stage design is proposed by implementing a proportional density index algorithm. For a multidimensional computerized adaptive testing model, a four-quadrant online calibration pretest item selection design with proportional density index algorithm is proposed. Comparisons were made between different online calibration item selection strategies. Results showed that under unidimensional computerized adaptive testing, the pro- posed modified two-stage item selection criterion with proportional density algorithm outperformed the other existing methods in terms of item parameter calibration and item parameter drift detection, and un- der multidimensional computerized adaptive testing, the online (re)calibration technique with the proposed four-quadrant item selection design with proportional density index outperformed other methods.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Psychology

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2016

Author dc:creator

• Guo, Rui

Contributors dc:contributor

• Chang, Hua-Hua
• Culpepper, Steven Andrew
• Douglas, Jeffrey A.
• Hubert, Lawrence J.
• Koehn, Hans-Friedrich

• Item parameter drift
• Online calibration
• Multidimensional item response theory
• Computerized adaptive testing

Statement dc:rights

• Copyright 2016 Rui Guo

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/90701

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/90701