Inconsistency in Assessment Center Performance: Measurement Error or Something More

The first assessment centers often considered inconsistency as a characteristic that might illuminate a candidate's personality and have important implications for future job performance (Bray & Grant, 1966; Fiske, Hanfmann, MacKinnon, Miller, & Murray, 1948), but subsequent assessment center researchers and practitioners have generally treated inconsistency as measurement error. However, increasing evidence suggests that participants may truly behave differently in different exercises (Lance et al., 2000; Lievens, 2001, 2002), and researchers in personality psychology have argued that consistency may be an individual difference, with some individuals displaying more consistency than others (e.g., Bern & Allen, 1974; Fleeson, 2001). The purpose of this dissertation is to explore empirically whether individual differences in the consistency of performance are measurable and meaningful in assessment center contexts. We present four studies that explore the existence of individual differences in performance consistency, both in assessment centers and in other settings, and the relationship between consistency and both personal and organizational outcomes. Study 1 addressed these questions in a general way by examining performance consistency in a large archival data set: college basketball statistics from the National Collegiate Athletic Association (NCAA). This exploratory study found that basketball players reliably differed in performance consistency and that consistency incrementally predicted both team and individual outcomes over and above average performance. In Study 2, an index of individual performance consistency appropriate for assessment centers and a corresponding index of differentiation across dimensions were proposed, and the behavior of these indices under varying conditions was explored via computer simulation. Study 3 then applied these indices to field data, analyzing ratings from two operational assessment centers and examining the distribution of the consistency and differentiation indices in the candidate population. Candidates reliably differed in consistency and, to a lesser extent, differentiation across dimensions. Study 4 analyzed ratings from another operational assessment center and found that the consistency index again incrementally predicted supervisor ratings of job performance, over and above average performance in the assessment center. Taken together, these studies provide a preliminary foundation for future investigations of performance consistency as an individual difference.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Psychology

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2015

Author dc:creator

• Gibbons, Alyssa Anne Mitchell

Contributors dc:contributor

• Rupp, Deborah E.

• Psychology, Industrial

Language dc:language

eng

Identifier

(MiAaPQ)AAI3269905

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/82128