The Impact Of Nested Testing On Experiment-Wise Type I Error Rate

<p>When conducting a statistical test the initial risk that must be considered is a Type I error, also known as a false positive. The Type I error rate is set by nominal alpha, assuming all underlying conditions of the statistic are met. Experiment-wise Type I error inflation occurs when multiple tests are conducted overall for a single experiment. There is a growing trend in the social and behavioral sciences utilizing nested designs. A Monte Carlo study was conducted using a two layer design. Five theoretical distributions and four real datasets taken from Micceri (1989) were used, each with five different samples sizes and conducted with nominal alpha set to 0.05 and 0.01. These were conducted both unconditionally and conditionally. All permutations were executed for 1,000,000 repetitions. It was found that when conducted unconditionally, the experiment-wise Type I error rate increases from alpha = 0.05 to 0.10 and 0.01 increases to 0.02. Conditionally, it is extremely unlikely to ever find results for the factor, as it requires a statistically significant nest as a precursor, which leads to extremely reduced power. Hence, caution should be used when interpreting nested designs.</p>

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

Open Access Dissertation

Discipline thesis:degree_discipline

Education Evaluation and Research

Year dc:date.available

2014

Author dc:creator

• Sawilowsky, Jack

Contributors dc:contributor

• Barry Markman

• Bonferroni-Dunn
• Experiment-wise Type I error inflation
• Hierarchical linear modeling
• Monte Carlo simulation
• Nested testing
• Educational Assessment, Evaluation, and Research
• Statistics and Probability

Repository record dc:identifier

https://digitalcommons.wayne.edu/oa_dissertations/914

OAI identifier oai:identifier

oai:digitalcommons.wayne.edu:oa_dissertations-1913