An in-depth analysis of DIF detection methods and equating techniques for valid test scores

Test equating is a key process in psychometric assessments, enabling the comparison of scores from different test forms intended to measure the same construct. This paper thoroughly examines the equating process within the framework of the Non-Equivalent groups with Anchor Test (NEAT) design. The NEAT design relies heavily on anchor items, which serve as a common reference across various test forms, aiding the comparative evaluation of test-taker performance. Chapter 2 explores anchor items and their functionality over administrations, emphasizing the implications of Differential Item Functioning (DIF) within the anchor set. A thorough examination of the impact of DIF items within the anchor set highlighted how they negatively affect biases and root mean square errors of test equating. The analysis stressed the crucial need for maintaining the psychometric consistency of anchor items, as any deviation can result in inaccurate equating outcomes and potentially unfair score comparisons across different test forms. Considering the ramifications of DIF items, Chapter 3 delves into another crucial facet of test equating – the detection of genuine DIF items. This chapter thoroughly studies five conventional DIF detection methods, evaluating their performance under different conditions. The comparative analysis found their effectiveness in identifying DIF items within the anchor item set to be condition-dependent, with each method's power and Type I error rates varying. This revelation emphasized the need for robust DIF detection mechanisms. Such mechanisms are crucial for identifying outliers, preserving the integrity of the test equating process, and ensuring fair and accurate outcomes. Transitioning to a critical stage of handling detected DIF items within the anchor set, Chapter 4 evaluates three strategies—ignoring, replacing, and removing DIF items. Each approach presents unique advantages and potential challenges. An in-depth analysis found that removing and replacing DIF items generally outperformed ignoring them, but this performance varied based on the DIF proportion, revealing certain drawbacks. The study highlighted the inherent trade-offs within each approach, underscoring the necessity for context-aware and careful management of DIF items. The findings enhance our understanding of test equating and contribute to the refinement of methodologies for ensuring fair and valid test score comparisons.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Educational Psychology

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2023

Author dc:creator

• Kartal, Gamze

Contributors dc:contributor

• Zhang, Jinming
• Chang, Hua-hua
• Anderson, Carolyn J
• Xia, Yan

• Test Equating
• Neat
• Anchor Items
• Differential Item Functioning
• Matching Variable
• Treatment Methods Of Differential Item Functioning

Statement dc:rights

• Copyright 2023 Gamze Kartal

Language dc:language

en, eng

Handle dc:identifier

https://hdl.handle.net/2142/121232

OAI record oai:identifier

https://www.ideals.illinois.edu/oai-pmh?verb=GetRecord&metadataPrefix=etdms&identifier=oai%3Awww.ideals.illinois.edu%3A2142%2F121232