Sensitivity analysis for publication bias in meta-analyses

This website conducts sensitivity analyses for publication bias in meta-analyses. Please use the following citation:

Mathur MB & VanderWeele TJ (2020). Sensitivity analysis for publication bias in meta-analyses. Journal of the Royal Statistical Society: Series C, 69(5), 1091-1119.

These analyses consider publication bias that favors affirmative studies (i.e., those with positive estimates and significant p-values) over nonaffirmative studies (i.e., those with negative or nonsignificant estimates). These analyses enable statements such as: “For publication bias to shift the observed point estimate to the null, significant results would need to be at least 30-fold more likely to be published than negative or nonsignificant results.” Alternatively, you can consider shifting to a chosen non-null value or shifting the confidence interval to include the null or another value.

This website also provides a worst-case meta-analytic point estimate under maximal publication bias obtained simply by conducting a standard meta-analysis of only the negative and nonsignificant studies.

Bias-corrected pooled estimate

For a chosen ratio of publication probabilities (selection ratio), estimate a publication bias-corrected pooled point estimate and 95% confidence interval.


Severity of publication bias needed to “explain away” results

Estimate the S-value, defined as the severity of publication bias (i.e., the ratio by which affirmative studies are more likely to be published than nonaffirmative studies, or vice versa) that would be required to shift the pooled point estimate or its confidence interval limit to q.


Significance funnel plot

The estimate among only nonaffirmative studies (gray diamond) represents a corrected estimate under worst-case publication bias that favors affirmative results. If the gray diamond represents a negligible effect size or if it is much smaller than the pooled estimate among all studies (black diamond), this suggests that the meta-analysis may not be robust to extreme publication bias.