The Development of a Hybrid Scoring Key for a Situational Judgment Test Designed for Training Evaluation

As a low fidelity work simulation, Situational Judgment Tests (SJTs) are an affordable and practical way of empirically linking training and on-the-job performance, thereby providing a viable means of evaluating training effectiveness. An issue, when utilizing SJTs, is deciding the appropriate manner in which the SJT should be scored. Traditional SJT scoring methodologies, while successfully utilized for selection and prediction, pose specific challenges when applied to a SJT designed to evaluate the effectiveness of a training program. This study discusses the shortcomings of traditional SJT scoring methodologies when used in the evaluation context. To overcome these challenges, an innovative scoring methodology, the Hybrid methodology, is presented. This study provides the detailed description of the Hybrid scoring key creation, and compares the Hybrid scoring key with two traditional scoring keys (Subject Matter Expert (SME) and Respondent-based scoring keys). Responses from a military training program are utilized to illustrate the distinctive effects of using the three different scoring approaches. The superiority of the hybrid scoring key, due to increased confidence in the key's accuracy, and findings regarding training evaluation are discussed. Future research directions and practical applications of the research are also discussed.

Name thesis:degree_name

Master of Science

Level thesis:degree_level

masters

Discipline thesis:degree_discipline

Psychology

Department dc:contributor.department

Psychology

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

2007

Author dc:creator

• Findlay, Rolanda A.

Chair dc:contributor.committeechair

• Hauenstein, Neil M. A.

Committee members dc:contributor.committeemember

• Cooper, Lee D.
• Foti, Roseanne J.

• Scoring Key
• Training Evaluation
• Situational Judgment Test

Statement dc:rights

• In Copyright

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Dc Identifier Other

etd-05182007-214843

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/42712