Random effects for variance estimates for meta-analysis

In performing a meta-analysis, treatment estimates from the individual trials are weighted

proportionately to the inverse of the variance of the contrast. In the case of a fixed effects analysis

the variance is purely internal and in the case of a random effects analysis a between-trials

variance term is added to the internal variance but the general principle is the same. Such

weighting is 'optimal' in some sense provided that the correct variances are used. However, in

practice the variances are not known and estimates are used instead. Where estimates are used,

however, the weighting is not optimal. Unfortunately the resulting meta-analysis estimate not only

has an overall true variance that is greater than that belonging to the optimal solution it also has a

reported variance that is less than that belonging to the optimal solution. In consequence

significance tests associated with it are too liberal and confidence intervals do not have correct

coverage properties. This is related to a general difficulty in the analysis of mixed effects models

using Reduced Maximum Likelihood (REML) and is a fact that has been known for nearly 70 years

but which is commonly ignored.

One solution is to consider a random effect on the variance. This, however, is not easy to

implement. One possible approach may be that of double hierarchical generalised linear models

(DHGLM) of Lee and Nelder. Another is to consider a two stage analysis in which shrunk

estimates of the variances are produced in a first stage using a random effects model on the

variance and then used to calculate weights for the meta-analysis itself. Alternatively, in principle,

a fully Bayesian implementation of an analysis should be possible using Monte-Carlo Markov

chain approaches.

In this project various approaches to this problem will be developed or investigated and compared.

The issue of random effects on variances also has relevance for sample size estimation and

adaptive designs and this is a possible direction the project could take.

References

Lee Y, Nelder JA. Double hierarchical generalized linear models. Journal of the Royal Statistical

Society Series C-Applied Statistics 2006;55:139-167.

Kenward MG, Roger JH. Small sample inference for fixed effects from restricted maximum

likelihood. Biometrics 1997;53(3):983-997.

Senn SJ. The many modes of meta. Drug Information Journal 2000;34:535-549.

Yates F, Cochran WG. The analysis of groups of experiments. Journal of Agricultural Science

1938;28(4):556-580.

Back to Stephen Senn's Research Page