Formulating a Dimtest-Based Effect Size Measure and a New DIMTEST Statistic and Their Applications

This dissertation proposes an interpretable effect size measure for use with DIMTEST, a widely used method for testing the hypothesis of test unidimensionality as represented by local item independence. There are six chapters in this dissertation. Chapter 1 addresses the importance of assessing unidimensionality and provides a review of the literature and methods. Chapter 2 reviews basic concepts of DIMTEST and the DIMTEST procedures for a statistical test of the null hypothesis. In Chapter 3 a DIMTET-based Effect Size Measure (DESM) is proposed. This chapter also evaluates the efficacy of DESM and the amount of bias in the DESM estimator by comparing it with the DESM population parameter. The results indicate that the DESM estimates converge to DESM parameters as test length and sample size increase. Even though DESM exhibits statistical bias with shorter tests, the amount of the bias decreases as test length and PT size increase. Chapter 4 of the dissertation develops a new DIMTEST statistic (NEWDIM) that yields improved Type I error rates relative to the original DIMTEST T statistic. Furthermore, this chapter evaluates the performances of DESM and NEWDIM in terms of Type I error and power. DESM used in tandem with DIMTEST results in better Type I error rates compared to using DIMTEST alone while having approximately equal power for cases of moderate multidimensionality. To better understand the effectiveness of DESM, NEWDIM, and DIMTEST, Chapter 5 focuses on short tests where DIMTEST has shown unexpectedly high Type I error rates. This section simulates short tests with total items less than 11 and compares the performance of DESM, NEWDIM, and DIMTEST under both unidimensional and multidimensional conditions. As expected, DIMTEST has Type I error inflation; however, using either DIMTEST or NEWDIM in tandem with DESM reduces greatly Type I error rates. Finally, the last chapter of the dissertation applies DESM and NEWDIM to real data in order to gain insight about their performances in practice.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Dissertation

Discipline thesis:degree_discipline

Educational Psychology

Grantor

University of Illinois at Urbana-Champaign

Year dc:date

2015

Author dc:creator

• Seo, Min Hee

Contributors dc:contributor

• Chang, Hua-Hua

• Education, Tests and Measurements

Language dc:language

eng

Identifier

(MiAaPQ)AAI3347507

OAI identifier oai:identifier

oai:www.ideals.illinois.edu:2142/80077