An examination of several in-basket scoring strategies and their effect on reliability and criterion-related validity

The in-basket exercise, a paper-and-pencil measure of administrative ability, is an assessment technique characterized by complex, often subjective scoring procedures which have limited wide-scale applications of this popular instrument. Because the literature affords no clear classification system for this and related exercises, a structure which clarified the definitions of and relationships among management games, management simulations, work sample tests, and in-basket exercises was introduced. The primary purpose of the present research was to investigate the cross-sample generalizability of several strategies for scoring the in-basket exercise, including a reduced-item scoring approach in which an optimal subset of items was identified for scoring. The preliminary studies upon which the present research is based are also discussed in the present work. In addition to examining the impact of these scoring strategies on reliability and criterion-related validity, consideration was given to addressing long-standing concerns of in-basket training- and scoring time demands. Three hundred and twenty-one entry-level employees from a large western Canadian utility company were administered the same in-basket exercise previously applied in a different Canadian utility company. Contrary to expectations, the shrinkage in validity using an empirically-based scoring key was substantial, pointing to the selection of a more logically-derived panel key as the method of choice. The introduction of a new cognitive-based measure of in-basket performance showed promising results. In addition, the reduced-item scoring approach did not result insignificant losses in reliability or criterion-related validity, thus allowing substantial reductions in training- and scoring-time. Implications for in-basket development are also considered.

Name thesis:degree_name

Master of Arts - MA

Level thesis:degree_level

master's

Discipline thesis:degree_discipline

Psychology

Grantor dc:publisher

University of British Columbia

Year dc:date

1992

Author dc:creator

• Harlos, Karen P.

Statement dc:rights

• For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.

Language dc:language

eng

Handle dc:identifier

http://hdl.handle.net/2429/1373

OAI identifier oai:identifier

oai:circle.library.ubc.ca:2429/1373