Accuracy of Global Fit Indices as Indictors of Multidimensionality in Multidimensional Rasch Analysis

Most research on confirmatory factor analysis using global fit indices (AIC, BIC, AICc, and CAIC) has been in the structural equation modeling framework. Little research has been done concerning application of these indices to item response models, especially within the framework of multidimensional Rasch analysis. The results of two simulations studies that investigated how sample size, between-dimension correlation, and test length affect the accuracy of these indices in model recovery using a multidimensional Rasch analysis are described in this dissertation. The first study analyzed dichotomous data, with model-to-data misfit as an additional independent variable. The second study analyzed polytomous data, with rating scale structure as an additional independent variable. The interaction effect between global fit index and between-dimension correlation had very large effect sizes in both studies. At higher values of between-dimension correlation, AIC indicated the correct two-dimension generating structure slightly more often than does the BIC or CAIC. The correlation by test length interaction had an odds ratio indicating practical importance in the polytomous study but not the dichotomous study. The combination of shorter tests and higher correlations resulted in a difficult-to-detect distinction being modeled with less statistical information. The correlation by index interaction in the dichotomous study had an odds ratio indicating practical importance. As expected, the results demonstrated that violations of the Rasch model assumptions are magnified at higher between-dimension correlations. Recommendations for practitioners working with highly correlated multidimensional data include creating moderate length (roughly 40 items) instruments, minimizing data-to-model misfit in the choice of model used for confirmatory factor analysis (MRCMLM or other MIRT models), and making decisions based on multiple global indices instead of depending on one index in particular.

Name thesis:degree_name

Ph. D.

Level thesis:degree_level

doctoral

Discipline thesis:degree_discipline

Educational Leadership and Policy Studies

Department dc:contributor.department

Educational Leadership and Policy Studies

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

2009

Author dc:creator

• Harrell, Leigh Michelle

Chairs dc:contributor.committeechair

• Skaggs, Gary E.
• Wolfe, Edward W.

Committee members dc:contributor.committeemember

• Terrell, George R.
• Miyazaki, Yasuo

• confirmatory factor analysis
• Rasch analysis
• model selection
• global fit indices
• BIC
• AIC

Statement dc:rights

• In Copyright

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Dc Identifier Other

etd-11182009-110003

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/29656