A new study by Ignacio Atal, a Long-term Fellow at CRI Research, and collaborators from Hôpital Hôtel-Dieu, just appeared in the Journal of Clinical Epidemiology. It shows how the statistical significance of meta-analyses can be fragile if you modify single events in the studies included in the meta-analysis.
The full text of the study is available here
Meta-analyses inform clinical practice by summarising treatment effect using results from multiple trials. However, the statistical significance of such meta-analysis, may rely on the outcome of only a few patients from specific trials included. The study evaluates this effect by defining the Fragility Index of meta-analyses, simply a minimal number of changes in the patient outcome (e.g. cured vs. not cured), that would change the study conclusions. Among 906 meta-analyses considered, the median Fragility Index was 12, but for about 30% of studies it was 5 or less. This indicates overall fragility of the outcomes of meta-analysis to small changes in the specific studies considered.
The authors also created a website that allows other scientists to evaluate the fragility of their meta-analyses.