Is Hiring Fair and Accurate? Perceptions of Statistical and Practical Significance of Adverse Impact Indices

Historically, the impact ratio and ZD test are widely used by organizations and the courts to assess a type of employment discrimination called adverse impact. However, previous court decisions have been inconsistent in their application of these measures. To understand this inconsistency, the present two-part study examines (1) how 31 personnel selection and legal experts select, apply, and communicate adverse impact measures in practice in a qualitative study, and (2) how 23 highly numerate experts make decisions about meaningful hiring differences when presented with adverse impact measures (the impact ratio and ZD test) in an experimental study. The qualitative study provides rich expert insights on the most generally recommended adverse impact measures, important situational and contextual factors, the measures most compelling when supporting adverse impact claims versus defending against claims, the easiest versus most difficult measures to communicate to stakeholders, and the measures viewed as ideal. Although many of the ideas discussed by the experts are not novel, some clear themes were identified in the qualitative study (i.e., statistical analyses, data, organization-centered factors, and contextual factors) as well as a host of subthemes (e.g., practical significance, sample size, data aggregation, communication, the goal of the adverse impact analysis). Turning to the experiment, all participants were highly numerate, creating range restriction that limited my original intent to analyze individual differences and decision-making quality through the lens of fuzzy-trace theory (a dual-process theory of memory and decision-making, e.g., Reyna & Brainerd, 1995). Nevertheless, I still found some preliminary support for Hypotheses 1-3, suggesting that participants extracted the meaning of the measures, and made calibrated ordinal judgments that were valid (distinguishing between no support to extreme support of meaningful hiring differences across conditions) and reliable (providing little variability in judgments within conditions). In regard to Hypothesis 4, both the impact ratio and ZD test were rated as similarly useful. Finally, an exploratory analysis suggested that experts relied more on the ZD test than the impact ratio and raw hiring rates when making judgments of meaningful hiring differences. I conclude with a two-part discussion that highlights integrative themes (i.e., history and legal precedent limiting the development and use of novel methods or improved practices, and communicating analyses to stakeholders) and elaborates on the experimental findings in light of fuzzy-trace theory.

Name thesis:degree_name

Master of Arts

Level thesis:degree_level

Masters

Discipline thesis:degree_discipline

Social Sciences

Grantor

Rice University

Year dc:date.issued

2024

Author dc:creator

• Courey, Karyssa A

Advisor dc:contributor.advisor

• Oswald, Frederick L

• adverse impact
• statistical significance
• practical significance
• statistical communication
• fuzzy-trace theory

Statement dc:rights

• Copyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder.

Language dc:language.iso

eng

Handle dc:identifier.uri

https://hdl.handle.net/1911/116123

OAI identifier oai:identifier

oai:repository.rice.edu:1911/116123