Differential Prediction: Understanding a Tool for Detecting Rating Bias in Performance Ratings

Three common methods have been used to assess the existence of rating bias in performance ratings: the total association approach, the differential constructs approach and the direct effects approach. One purpose of this study was to examine how the direct effects approach, and more specifically differential prediction analysis, is more useful than the other two approaches in examining the existence of rating bias. However, the usefulness of differential prediction depends on modeling the full rater race X ratee race interaction. Therefore, the second purpose of this study was to examine the conditions where differential prediction has sufficient power to detect this interaction. This was accomplished using monte carlo simulations. Total sample size, magnitude of rating bias, validity of predictor scores, rater race proportion and ratee race proportion were manipulated to identify which conditions of these parameters provided acceptable power to detect the rater race X ratee race interaction; in the conditions where power levels are acceptable, differential prediction is a useful tool in examining the existence of rating bias. The simulation results suggest that total sample size, magnitude of rating bias and rater race proportion have the most impact on power levels. Furthermore, these three parameters interact to effect power. Implications of these results are discussed.

Name thesis:degree_name

Master of Science

Level thesis:degree_level

masters

Discipline thesis:degree_discipline

Psychology

Department dc:contributor.department

Psychology

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

2008

Author dc:creator

• Tison, Emilee B.

Chair dc:contributor.committeechair

• Hauenstein, Neil M. A.

Committee members dc:contributor.committeemember

• Foti, Roseanne J.
• Stephens, Robert S.

• Rating Bias
• Differential Prediction
• Performance Ratings
• Race Effects
• Intercept Bias

Statement dc:rights

• In Copyright

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Dc Identifier Other

etd-03262008-102743

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/31549