REMI 1930E. Meta-analysis with erroneous conclusions: size matters

Artículo original: Random error in cardiovascular meta-analyses: how common are false positive and false negative results? AlBalawi Z, McAlister FA, Thorlund K, Wong M, Wetterslev J. Int J Cardiol 2013; 168(2): 1102-1107. [Summary] [Related articles]
Introduction: Meta-analysis is a set of statistical tools to provide a quantitative synthesis of the evidence, obtained from an exhaustive and explicit search ("systematic review"). It is recognised that a meta-analysis can not be insightful if evidence is scarce or poor-quality, so its results should not be interpreted in a mechanistic way. Bias in the primary studies can cause systematic errors, so risk of bias must be thoroughly and carefully evaluated. However, an aspect hitherto little considered in meta-analysis methodology is the likelihood to commit random error (type I error, to find statistically significant results by chance, and type II error, not finding significant results by chance) [1-3]. This risk is greater when the number of studies, patients and/or events is scarce or when data are subjected to repeated statistical testing, but the magnitude and implications of this problem are not well understood.
Summary: The authors evaluated all the reviews published by the cardiology group of the Cochrane Collaboration in 2012 including one or more meta-analysis of at least 5 randomized clinical trials. Meta-analyses were classified as true positives if their pooled sample size and/or their cumulative Z-curve crossed the O'Brien-Fleming monitoring boundaries for detecting a relative risk reduction (RRR) of at least 25%, and true negatives if their pooled sample size was sufficient to reject a RRR of 25%. Meta-analysis evaluated were 56, 23 with positive results, and 33 with negative results. 17% were considered potentially false positive and 64% potentially false negative results. Overall, 45% of the meta-analysis contained insufficient information to conclusively rule out a different effect of found. In most cases, the authors of the meta-analysis did not recognize this limitation of their results.
Comment: The study shows the risk of reaching premature conclusions with meta-analysis, and how even meta-analysis from an organization as rigorous as the Cochrane Collaboration fail to take into account in many cases the degree of uncertainty in the interpretation of their results. A clear example is that of hypothermia in cardiac arrest, whose effectiveness was considered proven prematurely [4-5]. For a correct interpretation of meta-analysis results is necessary to incorporate techniques to quantify the risk of random error.
Eduardo Palencia Herrejón
Hospital Universitario Infanta Leonor, Madrid.
© REMI, January 2014.
  1. Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta-analyses. Brok J, Thorlund K, Gluud C, Wetterslev J. J Clin Epidemiol 2008; 61: 763-769. [PubMed]
  2. Updating meta-analyses leads to larger type I errors than publication bias. Borm GF, Donders AR. J Clin Epidemiol 2009; 62: 825-830. [PubMed]
  3. Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. Wetterslev J, Thorlund K, Brok J, Gluud C. J Clin Epidemiol 2008; 61: 64-75. [PubMed]
  4. Hypothermia for neuroprotection in adults after cardiopulmonary resuscitation. Arrich J, Holzer M, Havel C, Mullner M, Herkner H. Cochrane Database Syst Rev 2012: CD004128. [PubMed]
  5. Hypothermia after cardiac arrest should be further evaluated--a systematic review of randomised trials with meta-analysis and trial sequential analysis. Nielsen N, Friberg H, Gluud C, Herlitz J, Wetterslev J. Int J Cardiol 2011; 151: 333-341. [PubMed]
  PubMed Search:
  • Enunciado: Relevance of information size on error risk in meta-analysis
  • Sintaxis: "information size" AND "meta-analysis as topic"[mh]
  • [Results]

No hay comentarios:

Publicar un comentario

Se ruega que los autores de los comentarios se identifiquen (nombre, apellidos, lugar de trabajo)

REMI 2353. Desmopresina para la disfunción plaquetaria del trauma craneoencefálico grave

ARTÍCULO ORIGINAL:  Desmopressin is a transfusion sparing option to reverse platelet dysfunction in patients with severe traumatic brain ...