Random effects for variance estimates for meta-analysis
In performing a meta-analysis, treatment estimates from the individual trials are weighted
proportionately to the inverse of the variance of the contrast. In the case of a fixed effects analysis
the variance is purely internal and in the case of a random effects analysis a between-trials
variance term is added to the internal variance but the general principle is the same. Such
weighting is 'optimal' in some sense provided that the correct variances are used. However, in
practice the variances are not known and estimates are used instead. Where estimates are used,
however, the weighting is not optimal. Unfortunately the resulting meta-analysis estimate not only
has an overall true variance that is greater than that belonging to the optimal solution it also has a
reported variance that is less than that belonging to the optimal solution. In consequence
significance tests associated with it are too liberal and confidence intervals do not have correct
coverage properties. This is related to a general difficulty in the analysis of mixed effects models
using Reduced Maximum Likelihood (REML) and is a fact that has been known for nearly 70 years
but which is commonly ignored.
One solution is to consider a random effect on the variance. This, however, is not easy to
implement. One possible approach may be that of double hierarchical generalised linear models
(DHGLM) of Lee and Nelder. Another is to consider a two stage analysis in which shrunk
estimates of the variances are produced in a first stage using a random effects model on the
variance and then used to calculate weights for the meta-analysis itself. Alternatively, in principle,
a fully Bayesian implementation of an analysis should be possible using Monte-Carlo Markov
In this project various approaches to this problem will be developed or investigated and compared.
The issue of random effects on variances also has relevance for sample size estimation and
adaptive designs and this is a possible direction the project could take.
Lee Y, Nelder JA. Double hierarchical generalized linear models. Journal of the Royal Statistical
Society Series C-Applied Statistics 2006;55:139-167.
Kenward MG, Roger JH. Small sample inference for fixed effects from restricted maximum
likelihood. Biometrics 1997;53(3):983-997.
Senn SJ. The many modes of meta. Drug Information Journal 2000;34:535-549.
Yates F, Cochran WG. The analysis of groups of experiments. Journal of Agricultural Science
Back to Stephen Senn's Research Page