Student-Group Item Parameter Drift Impact on Individual and Aggregate Observed Scores

This study was conducted to further research into the impact of student-group item parameterdrift (SIPD) —referred to as subpopulation item parameter drift in previous research— on abilityestimates and proficiency classification accuracy when occurring in the discrimination parameterof a 2-PL item response theory (IRT) model. Using Monte Carlo simulation, several factors weremanipulated and examined for impact including (1) direction of drift within the discriminationparameter, (2) true ability of student-groups, and (3) relative size of student-groups. ANOVA,including main effects and interaction effects, is used to evaluate bias in individual abilityestimates, aggregate ability estimates, bias in percent proficient, and change in percent proficient.Results indicated that SIPD occurring in the discrimination parameter does impact bias in severalways; most substantially in percent proficient classification accuracy but less substantially onindividual and aggregate ability estimates. Together, with findings from previous research aboutthe impact of this phenomenon in the difficulty parameter, SIPD should be a concern for statelevel assessment and accountability decision makers as it has the potential to lead to inaccuratedecisions regarding student performance. Suggestions for next steps when SIPD is suspected areprovided.

Grantor dc:publisher

University of Kansas

Year dc:date.issued

2023

Author dc:creator

• Hess, Jessica

Advisor dc:contributor.advisor

• Kingston, Neal

• Educational psychology
• Educational tests & measurements
• ability estimation
• accountability
• classification accuracy
• item parameter drift
• student-groups

Statement dc:rights

• Copyright held by the author.

Language dc:language.iso

en

Dc Identifier Other

http://dissertations.umi.com/ku:18818

OAI identifier oai:identifier

oai:kuscholarworks.ku.edu:1808/38858