Evaluating the robustness of FlexMIRT on DIF analysis

Item Response Theory (IRT) plays a crucial role in educational measurement, and accurately detecting Differential Item Functioning (DIF) items is critical to ensuring fairness and validity in tests. This study focused on flexMIRT, a software for multilevel and IRT analysis, performance in DIF analysis within the three-parameter logistic (3PL) and four-parameter logistic (4PL) models. Specifically, the aim was to evaluate the robustness of flexMIRT when applied to the 4PL model, which was not the software's true model. Simulation studies evaluated the ability to control Type I error rates and detect DIF items with varying sample sizes, test lengths, percentages of DIF items, and different levels of parameter change. The results of the simulation studies indicated that flexMIRT maintains low Type I error rates for discrimination (a) and difficulty (b) parameters across both models. An uptick in Type I error rates for the test of guessing parameter (g) was noted as the sample size increased. For Type II error rates, flexMIRT demonstrated enhanced detection capabilities in larger samples and with significant parameter changes, affirming its effectiveness in different IRT models. While flexMIRT showed adaptability in DIF detection, careful interpretation was warranted for the g parameter in extensive datasets. Future research directions include evaluating flexMIRT's robustness with IRT models, exploring a more comprehensive array of testing conditions, and establishing a more definitive framework for detecting significant DIF to provide precise usage guidelines for flexMIRT.

Name thesis:degree_name

M.S.

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Educational Psychology

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2023

Author dc:creator

• Liu, Yiqing

Contributors dc:contributor

• Zhang, Jinming
• Xia, Yan
• Zhang, Susu

• Item Response Theory
• Differential Item Functioning
• Flexmirt

Statement dc:rights

• Copyright 2023 Yiqing Liu

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/122179

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F122179