Power approximation for the test of study-level categorical moderators in meta-regression with dependent effect sizes

Sample size and statistical power are key considerations when planning a research synthesis. While power analysis methods for the tests of moderators have been established for fixed- and random-effects models for independent effects, there is currently no methodology for conducting power analysis for moderator tests in meta-regression models that account for dependence. Building on a previous study that evaluated power approximations for the test of an average effect size (Vembye et al., 2023), I propose a new approximation formula specifically for testing study-level categorical moderators using the correlated-hierarchical effects model with robust variance estimation (CHE+RVE). Additionally, I conduct a Monte Carlo simulation to validate this power approximation formula against the true simulated power of a test of multiple contrasts from a CHE+RVE model. I also examine the Type I error rates and power of a test of multiple contrasts corrected for small samples from a CHE+RVE model. The results from my study show that the power approximation formula is accurate when there is a small number of contrasts, but it could be inaccurate in conditions with a larger number of contrasts and small degrees of freedom. Additionally, I replicate past findings that the small-sample adjusted test of multiple contrasts using RVE is conservative when there is a higher number of contrasts and a small number of studies.

Name thesis:degree_name

Doctor of Philosophy

Discipline thesis:degree_discipline

Educational Psychology

Grantor

The University of Texas at Austin

Year dc:date.issued

2025

Author dc:creator

• Bhat, Bethany Hamilton

Advisors dc:contributor.advisor

• Beretvas, Susan Natasha
• Pustejovsky, James E.

Committee members dc:contributor.committeemember

• Xiao Liu
• Tiffany A. Whittaker
• Therese D. Pigott

• Robust variance estimation
• Categorical moderators
• Meta-analysis
• Dependent effect sizes
• Power
• Study-level moderators
• Meta-regression

Language dc:language.iso

English

Identifier URI

https://doi.org/10.26153/tsw/62115

OAI identifier oai:identifier

oai:repositories.lib.utexas.edu:2152/134793