Application of Appropriateness Measurement to a Problem in Adaptive Testing

The effects of an initial sequence of 1-5 random responses to 15, 20 and 25-item adaptive tests were examined using simulation. Results indicated that adaptive tests are not robust to initial sequences of misinformative responses. Significant underestimation of ability occurred when as few as the initial 2 responses were random. A likelihood ratio (LR) statistic was used to detect the occurrence of this type of response aberrance. The statistic was evaluated by determining hit rates and false positive rates under several conditions, including: (1) comparison with other appropriateness indices; (2) standardization of the likelihood ratio; and (3) cases where the alternative hypothesis specified in the likelihood ratio was not the actual form of response aberrance. The LR index detected initial sequences of random responses with high levels of accuracy. Other appropriateness indices provided considerably lower detection rates. Possible explanations for these findings are presented and areas for future research are identified.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Psychology

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2014

Author dc:creator

• Candell, Gregory Lloyd

• Psychology, Psychometrics

Identifier

(UMI)AAI8908637

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/69713