Does Item Exposure Impact the Relationship Between Specificity and Interrater Agreement

The present study attempted to replicate the results found in Roch and Paquin (2004). Raters in their study viewed a video tape performance and rated a target's performance during a leaderless group discussion. Their results indicated that as item specificity decreased (i.e., as items became more vague), there was an increase in interrater agreement, a positive correlation between item difficulty and interater agreement, and a positive correlation between performance and interrater agreement. The present study also examined the impact of item exposure as it related to these hypotheses and the relationship between item exposure and interrater agreement. Using a sample of 299 participants, the present study used the same video of a leaderless group discussion, rating form, and specificity ratings as used in Roch and Paquin (2004). The results of this study found a positive correlation between specificity and agreement and that item exposure significantly positively impacted interrater agreement.

Name thesis:degree_name

Master of Arts

Discipline thesis:degree_discipline

Department of Psychology

Year

2006

Author dc:creator

• Littlejohn, Traxler

• Psychology

Repository record dc:identifier

https://digitalcommons.wku.edu/theses/292

OAI identifier oai:identifier

oai:digitalcommons.wku.edu:theses-1295