Holistic versus Decomposed Rating Scales: Which causes higher levels of cognitive load?

The purpose of this study was to help explore the assumptions in job analysis literature involving whether holistic or decomposed job analysis items lead to a higher level of cognitive load for raters. The main study, involving 303 undergraduate university students, was a 2 (Type of measure: holistic or decomposed) X 2 (Level of extraneous cognitive load: additional load or no additional load) within-subjects design. The 160 decomposed items analyzed in this study were pilot tested to ensure that they would correlate with the 17 holistic items. Under the additional cognitive load condition, participants memorized an 8-digit number, and then were asked to recall and recognize this number upon the completion of the rating task (this manipulation was performed for both the holistic and decomposed measures). Stability of ratings across conditions and interrater agreement were used as dependent measures. Results indicated that the holistic items (r=.74) had higher levels of stability across cognitive load conditions than did the decomposed items (r=.66). The levels of interrater agreement were not significantly different between three of the four conditions. In partial support of Butler and Harvey (1988), the level of interrater agreement for the Holistic additional cognitive load condition (r*wg=.33) was significantly lower than the interrater agreement for the remaining three conditions. The pattern of results supported prior research (Cornelius & Lyness, 1980; Lyness & Cornelius, 1982) indicating that, depending on the criteria being used, holistic items do not necessarily cause a higher level of cognitive load for raters than do decomposed items.

Name thesis:degree_name

Master of Science

Level thesis:degree_level

masters

Discipline thesis:degree_discipline

Psychology

Department dc:contributor.department

Psychology

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

2003

Author dc:creator

• Watt, Alisa H.

Chair dc:contributor.committeechair

• Foti, Roseanne J.

Committee members dc:contributor.committeemember

• Finney, Jack W.
• Harvey, Robert J.
• Crawford, Helen J.

• Decomposed
• Holistic
• Job Analysis
• Cognitive load

Statement dc:rights

• In Copyright

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Dc Identifier Other

etd-05092003-081506

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/32420