The Influence of Direct Versus Indirect Observation, Candidate Report Format, and Assessor Training on the Accuracy of Assessor Ratings

<p>Within the assessment center method, assessors' preliminary evaluations of candidates' performance are typically based on information obtained through direct observation of the candidate's performance in the situational exercise, or on another assessor's report of that performance. This variation is somewhat disconcerting, however, in that its impact on assessor ratings remains largely unstudied. The primary focus of the present study was to assess the differential effects of observation type (i.e., direct observation, dimension-specific report, narrative report) on various measures of rating accuracy. In addition, the present study investigated cognitive modeling as an assessor training strategy, and its impact on rating accuracy. Seventy-three undergraduates majoring in business administration were either trained or not trained, and either observed and rated nine videotapes depicting individuals conducting performance review sessions, or reviewed and evaluated corresponding dimension-specific or narrative reports describing the same performance. A two (cognitive modeling training, no training) by three (direct observation, dimension-specific report, narrative report) analysis of variance design was used to assess the effects of training and observation type on rating accuracy.</p> <p>Moderate support was found for the hypotheses that direct observation would yield greater accuracy than report-based ratings. While ratings obtained in the direct observation condition were generally more accurate than narrative-report based ratings, dimension-specific report ratings produced superior accuracy. In addition, those participants receiving the cognitive modeling training, as predicted, were significantly more accurate in their ratings than the no-training participants.</p> <p>These results suggest that in assessment situations where assessors must rely on reports to evaluate candidate performance, those reports should be constructed in a dimension-specific format. Furthermore, cognitive modeling appears to be a viable strategy for assessor training. Future research should examine reports which more closely typify actual assessment center products and their effects on rating accuracy. The cognitive modeling approach to assessor training should also be given greater attention in further study.</p>

Name thesis:degree_name

Doctor of Philosophy (PhD)

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Psychology

Year dc:date.available

1987

Author dc:creator

• Johnson, Rudolph L., Jr.

Contributors dc:contributor

• Terry L. Dickinson
• Robert M. McIntyre
• Albert Glickman
• William Silverman

• Assessment centers
• Employee assessment
• Assessor training
• Industrial and Organizational Psychology

Statement dc:rights

• <p>In Copyright. URI: <a href="http://rightsstatements.org/vocab/InC/1.0/">http://rightsstatements.org/vocab/InC/1.0/</a> This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).</p>

Repository record dc:identifier

https://digitalcommons.odu.edu/psychology_etds/290

OAI identifier oai:identifier

oai:digitalcommons.odu.edu:psychology_etds-1285