Assessing the Interpretive Component of Criterion-Referenced Test Item Validity

The usefulness of an innovative testing technique for empirically detecting ambiguously worded or structurally deficient test items was explored. In addition to responding, eighty-nine undergraduate students in the architecture curriculum at the University of Illinois at Urbana-Champaign were asked to classify each test item on an electricity examination as having been generated from one or more of the topics representing the sub-unit headings from their text. These judgments were compared to their professor's "standard" judgments. For these data, an index of item-domain divergence in perceived item meaning between examinees on the average and the professor as content specialist was computed. Second, the amount of unexplained variation in response data not accounted for by classification data was computed for each "standard" domain label for each item. Third, the mean, variance, and n of total test scores for examinees was reported for each of the following examinee groups for each item: responded incorrectly, same classification; responded correctly, same classification; responded incorrectly, different classification; and responded correctly, different classification. Finally, z scores were computed for the difference between the mean total test scores for each of the above cells as compared to the mean total test score for all examinees for each "standard" domain for each item.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Education

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2014

Author dc:creator

• Secolsky, Charles

• Education, Tests and Measurements

Language dc:language

eng

Identifier

(UMI)AAI8026593

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/65968