Meta-analysis with a single study

Statistical Methods in Medical Research, vol. 34, pp. 2302–2312

Abstract

Effect sizes typically vary among studies of the same intervention. In a random effects meta-analysis, this source of variation is taken into account, at least to some extent. However, when we have only one study, the heterogeneity remains hidden and unaccounted for. Treating the study-level effect as if it is the population-level effect leads to underestimation of the uncertainty. We propose an empirical Bayesian approach to address this problem. We start by estimating the distribution of the population-level effects and heterogeneity among 1635 meta-analyses from the Cochrane Database of Systematic Reviews. Using both synthetic data and cross-validation, we assess the consequences of using these estimated distributions as prior information for the analysis of single trials. We find that our Bayesian "meta-analyses of single studies" perform much better than naively assuming non-varying effects. The prior on the heterogeneity results in better quantification of the uncertainty. The prior on the treatment effect substantially reduces the mean squared error both for estimating the study-level and population-level effects. For the latter, this reduction is equivalent to doubling the sample size.

Authors 3

1. Erik W. van Zwet corresponding Aachen Department of Biomedical Data Sciences

   Leiden University Medical Center

   Affiliation as printed

   Leiden University Medical Center

   Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands

2. Witold Więcek

   University of Chicago

   Affiliation as printed

   University of Chicago

   Development Innovation Lab, University of Chicago, Chicago, IL, USA

3. Andrew Gelman

   Columbia University

   Affiliation as printed

   Columbia University

   Department of Statistics and Department of Political Science, Columbia University, New York, NY, USA