Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 7 of 7 for “"rater effects"”.
-
Extended many-facet Rasch models: Accounting for rater effects in automated essay scoring systems
… and accounting for systemic error by human raters which can lead to the training of AES systems on biased training labels, resulting in bias in the trained AES system. Previous research has developed many-facet Rasch models (MFRMs) that can model and account for rater severity; however, …
-
Robustness of Inter-Rater Reliability Estimators to Rater Leniency/Severity Effects In An Absolute Decision Setting - A Monte Carlo Study of Performance Ratings
<p>Inter-rater reliability coefficients are often reported in performance assessment as a measure of rating quality. Rater effects, as a systematic error in rating, may change the decision made about an examinee when an absolute interpretation of scores, such as a pass/fail decision, is used. …
-
Dealing with Sparse Rater Scoring of Constructed Responses within a Framework of a Latent Class Signal Detection Model
… an examinee's response is evaluated by only one rater, which is called a single rater design. For example, in a classroom assessment practice, only one teacher grades each student's performance. While single rater designs are the most cost-effective method among all rater designs, the lack of a …
-
Analysis of Russian and Spanish Subskill testing at the Defense Language Institute
… is used identify Subskill tests with significant rater effects and the magnitude of those effects when they are present. Item Response Theory is used to examine the Subskill tests at the question level in order to identify questions that poorly discriminate between students of different abilities. …
-
The Relation between Uncertainty in Latent Class Membership and Outcomes in a Latent Class Signal Detection Model
… in the educational field, more specifically, rater issues involved in essay grading such as rater effects and rater reliability. An LC-SDT model has the potential for wide applications in education as well as other areas. Therefore it is important to explore the issue of accounting for …
-
Rater Drift in Constructed Response Scoring via Latent Class Signal Detection Theory and Item Response Theory
… As such, they require the judgment of multiple raters that are subject to differences in perception and prior knowledge of the material being evaluated. As with any scoring procedure, the scores assigned by raters must be comparable over time and over different test administrations and forms; in …
-
Detecting Rater Centrality Effect Using Simulation Methods and Rasch Measurement Analysis
This dissertation illustrates how to detect the rater centrality effect in a simulation study that approximates data collected in large scale performance assessment settings. It addresses three research questions that: (1) which of several centrality-detection indices are most sensitive to the …