Performance of the Kenward-Project when the Covariance Structure is Selected Using AIC and BIC

Linear mixed models are frequently used to analyze data with random effects and/or repeated measures. A common approach to such analyses requires choosing a covariance structure. Information criteria, such as AIC and BIC, are often used by statisticians to help with this task. However, these criteria do not always point to the true covariance structure and therefore the wrong covariance structure is sometimes chosen. Once this step is complete, Wald statistics are used to test fixed effects. Degrees of freedom for these statistics are not known. However, there are approximation methods, such as Kenward and Roger (KR) and Satterthwaite (SW) that have been shown to work well in some situations. Schaalje et al. (2002) concluded that the KR method would perform at least as well as or better than the SW method in many cases assuming that the covariance structure was known. On the other hand, Keselman et al. (1999) concluded that the performance of the SW method when the covariance structure was selected using AIC was poor for negative pairings of treatment sizes and covariance matrices and small sample sizes. Our study used simulations to investigate Type I error rates in test of fixed effects using Wald statistics with the KR adjustment method, incorporating the selection of the covariance structure using AIC and BIC. Performance of the AIC and BIC criteria in selecting the true covariance structure was also studied. The MIXED procedure (SAS v. 9) was used to analyze each simulated data set. Type I error rates from the best AIC and BIC models were always higher than target values. However, Type I error rates obtained by using the BIC criterion were better than those obtained by using the AIC criterion. Type I error rates for the correct models were often adequate depending on the sample size and complexity of covariance structure. Performance of AIC and BIC was poor. This could be a consequence of small sample sizes and the high number of covariance structures these criteria had to choose from.

Name thesis:degree_name

MS

Grantor dc:publisher

Brigham Young University - Provo

Author dc:creator

• Gomez, Elisa Valderas

• mixed models
• Kenward-Roger
• test of fixed effects
• Wald statistics
• selection criteria
• AIC
• BIC
• Statistics and Probability

Language dc:language

English

Repository record dc:identifier

https://scholarsarchive.byu.edu/etd/33

OAI identifier oai:identifier

oai:scholarsarchive.byu.edu:etd-1032