Comparing outcome measures derived from four research designs incorporating the retrospective pretest.

Over the last 5 decades, the retrospective pretest has been used in behavioral science research to battle key threats to the internal validity of posttest-only control-group and pretest-posttest only designs. The purpose of this study was to compare outcome measures resulting from four research design implementations incorporating the retrospective pretest: (a) pre-post-then, (b) pre-post/then, (c) post-then, and (d) post/then. The study analyzed the interaction effect of pretest sensitization and post-intervention survey order on two subjective measures: (a) a control measure not related to the intervention and (b) an experimental measure consistent with the intervention. Validity of subjective measurement outcomes were assessed by correlating resulting to objective performance measurement outcomes. A Situational Leadership® II (SLII) training workshop served as the intervention. The Work Involvement Scale of the self version of the Survey of Management Practices Survey served as the subjective control measure. The Clarification of Goals and Objectives Scale of the self version of the Survey of Management Practices Survey served as the subjective experimental measure. The Effectiveness Scale of the self version of the Leader Behavior Analysis II® served as the objective performance measure. This study detected differences in measurement outcomes from SLII participant responses to an experimental and a control measure. In the case of the experimental measure, differences were found in the magnitude and direction of the validity coefficients. In the case of the control measure, differences were found in the magnitude of the treatment effect between groups. These differences indicate that, for this study, the pre-post-then design produced the most valid results for the experimental measure. For the control measure in this study, the pre-post/then design produced the most valid results. Across both measures, the post/then design produced the least valid results.

Grantor dc:publisher

University of North Texas

Year dc:date

2007

Author dc:creator

• Nimon, Kim F.

Contributors dc:contributor

• Allen, Jeff M.
• Beyerlein, Michael
• Li, Jessica

• retrospecitve pretest
• vailidity
• self-reports
• response-shift
• pretest sensitization
• implicit theories
• Experimental design.
• Research -- Statistical methods.
• Leadership -- Study and teaching -- Evaluation.

Statement dc:rights

• Public
• Copyright
• Nimon, Kim
• Copyright is held by the author, unless otherwise noted. All rights reserved.

Language dc:language

English

Identifier

oclc: 192020292
https://digital.library.unt.edu/ark:/67531/metadc3931/
ark: ark:/67531/metadc3931

OAI identifier oai:identifier

info:ark/67531/metadc3931