Item selection methods in multidimensional computerized adaptive testing adopting polytomously-scored items under multidimensional generalized partial credit model

Four item selection methods are compared and investigated under three test formats in the context of Multidimensional Computerized Adaptive Testing (MCAT) delivering polytomous items partially or completely in tests. Item selection methods examined include Fisher information based D-optimality (D-optimality), Kullback-Leibler information index (KI), mutual information (MI), and continuous entropy method (CEM). The three test formats considered are the POLYTYPE format that contains polytomous items with three response categories, the DPMIX format that delivers dichotomous items at the beginning and polytomous items at the final stage, and the PDMIX format that has the reverse order as DPMIX. In general, D-optimality shows the best estimation accuracy and conditional estimation accuracy. D-optimality, MI, and CEM are similar in terms of ability estimation accuracy and tendency in selecting items when the item bank size is large. For both dichotomous and polytomous items, KI is mostly outperformed by the other three methods in terms of ability estimation precision.When sub-thetas in both dimensions are equal,however,KI shows the best performance for polytomous items. In this study, which item type, dichotomous or polytomous, being administered first does not affect the estimation accuracy. However, if the test length is much longer or shorter than the test length of the current study, it is possible that the estimation accuracy could be affected by the order of delivering different item types. Both DPMIX and PDMIX formats yield similar conditional estimation accuracy pattern and precision. In addition, the item bank size does affect the estimation precision. These conclusions, however, might not be applied to MCAT testing with different test designs or item pool structures. More studies are needed in MCAT combining with polytomous items to further facilitate the development and improvement of the next-generation assessments such as formative assessment or testing for diagnosis.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Educational Psychology

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2012

Author dc:creator

• Lin, Haiyan

Contributors dc:contributor

• Chang, Hua-Hua
• Ryan, Katherine E.
• Anderson, Carolyn J.
• Douglas, Jeffrey A.

• Multidimensional Computerized Adaptive Testing
• Item Selection Methods
• Multidimensional Item Response Theory
• Multidimensional Generalized Partial Credit Model

Statement dc:rights

• Copyright 2012 Haiyan Lin

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/34534

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/34534