Interest item diagnostics through IRT: an application of GPCM and tree-based IRT models to contemporary measures of interests

Item Response Theory (IRT) is a contemporary test evaluation technique that is frequently used in the fields of achievement and aptitude testing. However, IRT research within the field of vocational interest test evaluation is nascent at best. This dissertation advances the use of IRT in vocational interest through two studies of contemporary interest measures: The O*NET Interest Profiler (Rounds, Hoff & Lewis, 2021) and the Comprehensive Assessment of Basic Interests (CABIN; Su, Tay, Liao, Zhang & Rounds, 2018). For the O*NET Interest Profiler (IP), I use IRT to demonstrate the viability of a new tree-based response model for answering inventory items and compare the results of test shortening using the tree-based IRT model versus a more standard 2 Parameter Logistic Model. I found that the tree based model was more apt for describing the ‘Like/Dislike/Unsure’ response data of the O*NET IP and discovered that respondents tend to form similar thought processes when approaching these interest items—They make a two-step decision of whether or not they like the item first, before deciding if they actually dislike the item second. For the CABIN, I use a Generalized Partial Credit Model (GPCM) to obtain item diagnostics for the 8 interest subscales and propose a shortened version of each scale using item information curves and marginal reliability coefficients. I proposed two short forms of the CABIN based on these diagnostics—One that preserves the 41 basic interest scales of the original, and an even shorter version that simply measures the 8 overarching interest constructs. Researchers should select a short form that matches their desired level of specificity of the measured construct, and I illustrate how IRT can help in this endeavor. Both studies reveal the untapped potential IRT has for refining measures in vocational interest research and understanding the item response process of interest items.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Psychology

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2021

Author dc:creator

• Wee, Jian Ming Colin

Contributors dc:contributor

• Rounds, James B
• Newman, Daniel A
• Kern, Justin L
• Napolitano, Christopher M
• Hoff, Kevin A

• vocational interests
• item response theory
• psychometrics

Statement dc:rights

• Copyright 2021 Colin Wee Jian Ming

Language dc:language

en

Handle dc:identifier

http://hdl.handle.net/2142/110540

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/110540