Student-Group Item Parameter Drift Impact on Individual and Aggregate Observed Scores
type: CreativeWork
description: This study was conducted to further research into the impact of student-group item parameterdrift (SIPD) —referred to as subpopulation item parameter drift in previous research— on abilityestimates and proficiency classification accuracy when occurring in the discrimination parameterof a 2-PL item response theory (IRT) model. Using Monte Carlo simulation, several factors weremanipulated and examined for impact including (1) direction of drift within the discriminationparameter, (2) true ability of student-groups, and (3) relative size of student-groups. ANOVA,including main effects and interaction effects, is used to evaluate bias in individual abilityestimates, aggregate ability estimates, bias in percent proficient, and change in percent proficient.Results indicated that SIPD occurring in the discrimination parameter does impact bias in severalways; most substantially in percent proficient classification accuracy but less substantially onindividual and aggregate ability estimates. Together, with findings from previous research aboutthe impact of this phenomenon in the difficulty parameter, SIPD should be a concern for statelevel assessment and accountability decision makers as it has the potential to lead to inaccuratedecisions regarding student performance. Suggestions for next steps when SIPD is suspected areprovided.
abstract: This study was conducted to further research into the impact of student-group item parameterdrift (SIPD) —referred to as subpopulation item parameter drift in previous research— on abilityestimates and proficiency classification accuracy when occurring in the discrimination parameterof a 2-PL item response theory (IRT) model. Using Monte Carlo simulation, several factors weremanipulated and examined for impact including (1) direction of drift within the discriminationparameter, (2) true ability of student-groups, and (3) relative size of student-groups. ANOVA,including main effects and interaction effects, is used to evaluate bias in individual abilityestimates, aggregate ability estimates, bias in percent proficient, and change in percent proficient.Results indicated that SIPD occurring in the discrimination parameter does impact bias in severalways; most substantially in percent proficient classification accuracy but less substantially onindividual and aggregate ability estimates. Together, with findings from previous research aboutthe impact of this phenomenon in the difficulty parameter, SIPD should be a concern for statelevel assessment and accountability decision makers as it has the potential to lead to inaccuratedecisions regarding student performance. Suggestions for next steps when SIPD is suspected areprovided.
author: Hess, Jessica
language: en
keywords: Educational psychology, Educational tests & measurements, ability estimation, accountability, classification accuracy, item parameter drift, student-groups
date published: 2023-01-01
date modified: 2026-07-24T02:44:56Z
provider: University of Kansas
publisher: Networked Digital Library of Theses and Dissertations
type: WebPage
source organization: University of Kansas

PropertyValue
property ID: Handle
value: https://hdl.handle.net/1808/38858