Make Some Noise: Methods for Generating Data from Imperfect Factor Models

Researchers conducting Monte Carlo simulation studies involving covariance structure models (e.g., the common factor model) have increasingly recognized the importance of incorporating error due to model misfit in simulated data. Incorporating this model error acknowledges that all models are literally false, and no covariance structure model will fit perfectly in the population. Several methods for generating data from error-perturbed models have been proposed, including the Tucker, Koopman, and Linn (TKL; 1969), Cudeck and Browne (CB; 1992), and Wu and Browne (WB; 2015) model-error methods. All of these methods require user-specified parameter values that determine the degree of model misfit to be introduced. In particular, the CB and WB methods each have a single parameter that is related to the desired Root Mean Square Error of Approximation (RMSEA) value for the simulated covariance matrix. In contrast, the TLK method includes two parameter values that are generally chosen to align with values used in previous simulation studies or by testing many combinations of parameter values until solutions have RMSEA values that are close to the desired value. However, although RMSEA has often been used to indicate the degree of misfit introduced by model-error methods, RMSEA alone does not provide a complete summary of model fit. To get a more complete summary of model fit, other types of fit indices like the Comparative Fit Index (CFI) should also be used. Unfortunately, the TKL, CB, and WB model-error methods do not provide a way to specify multiple target fit indices. To address this issue, I proposed an optimization procedure that allows users to specify either a target RMSEA value, a target CFI value, or both simultaneously, and then attempts to find a combination of parameter values that produces a solution with fit indices close to the target values. To test the procedure, I conducted a simulation study using the proposed multiple-target TKL method, the CB method, and the WB method to generate error-perturbed correlation matrices for models with varying numbers of factors, items per factor, salient factor loadings, factor correlations, and target model fit indices. The results of the simulation study showed that the multiple-target TKL method was more likely than the alternative methods to lead to solutions with RMSEA and CFI values corresponding to similar qualitative levels of model fit. Moreover, the multiple-target TKL method produced solutions with RMSEA and CFI values that were closer to specified target RMSEA and CFI values than the alternative model-error methods. Thus, the multiple-target TKL method should be a useful tool for researchers who wish to generate error-perturbed correlation matrices in Monte Carlo simulation studies. To facilitate its use, I wrote an R package (noisemaker) with implementations of the multiple-target TKL method, along with implementations of the CB and WB model-error methods.

Author dc:creator

• Kracht, Justin

• adventitious error
• covariance structure models
• factor models
• model error

Language dc:language.iso

en

Handle dc:identifier.uri

https://hdl.handle.net/11299/250414

OAI identifier oai:identifier

oai:conservancy.umn.edu:11299/250414