Meta-analyses of Adverse Effects Data Derived from Randomised Controlled Trials as Compared to Observational Studies: Methodological Overview

BackgroundThere is considerable debate as to the relative merits of using randomised controlled trial (RCT) data as opposed to observational data in systematic reviews of adverse effects. This meta-analysis of meta-analyses aimed to assess the level of agreement or disagreement in the estimates of harm derived from meta-analysis of RCTs as compared to meta-analysis of observational studies. Methods and FindingsSearches were carried out in ten databases in addition to reference checking, contacting experts, citation searches, and hand-searching key journals, conference proceedings, and Web sites. Studies were included where a pooled relative measure of an adverse effect (odds ratio or risk ratio) from RCTs could be directly compared, using the ratio of odds ratios, with the pooled estimate for the same adverse effect arising from observational studies. Nineteen studies, yielding 58 meta-analyses, were identified for inclusion. The pooled ratio of odds ratios of RCTs compared to observational studies was estimated to be 1.03 (95% confidence interval 0.93–1.15). There was less discrepancy with larger studies. The symmetric funnel plot suggests that there is no consistent difference between risk estimates from meta-analysis of RCT data and those from meta-analysis of observational studies. In almost all instances, the estimates of harm from meta-analyses of the different study designs had 95% confidence intervals that overlapped (54/58, 93%). In terms of statistical significance, in nearly two-thirds (37/58, 64%), the results agreed (both studies showing a significant increase or significant decrease or both showing no significant difference). In only one meta-analysis about one adverse effect was there opposing statistical significance. ConclusionsEmpirical evidence from this overview indicates that there is no difference on average in the risk estimate of adverse effects of an intervention derived from meta-analyses of RCTs and meta-analyses of observational studies. This suggests that systematic reviews of adverse effects should not be restricted to specific study types. Please see later in the article for the Editors' Summary背景 目前，在不良反应系统评价（systematic review）中，相较于观察性数据（observational data），使用随机对照试验（randomised controlled trial, RCT）数据的相对优劣仍存在较大争议。本荟萃分析的荟萃分析（meta-analysis of meta-analyses）旨在评估：相较于观察性研究（observational study）的荟萃分析，基于随机对照试验荟萃分析得出的不良反应风险估计值的一致性程度。方法与结果 研究除通过参考文献核查、联系专家、引文检索以及手工检索核心期刊、会议论文集与网站外，还在10个数据库中开展了文献检索。纳入研究需满足：可直接通过比值比（odds ratio）之比，对比随机对照试验荟萃分析得出的某一不良反应的合并相对效应量（比值比或风险比（risk ratio）），与观察性研究得出的同一不良反应的合并效应量。最终共纳入19项研究，涵盖58项荟萃分析。经估算，随机对照试验与观察性研究的合并比值比之比为1.03（95%置信区间（confidence interval）：0.93~1.15）。研究规模越大，两者间的差异越小。对称的漏斗图（funnel plot）提示，基于随机对照试验数据的荟萃分析与基于观察性研究的荟萃分析所得到的风险估计值并无系统性差异。几乎所有情况下，不同研究设计的荟萃分析所得到的不良反应风险估计值的95%置信区间均存在重叠（54/58，93%）。在统计学显著性层面，近三分之二（37/58，64%）的结果一致：即两类研究均显示不良反应显著升高或降低，或均无统计学显著性差异。仅在1项针对某一不良反应的荟萃分析中，出现了统计学显著性相反的结果。结论 本荟萃分析的荟萃分析的实证证据表明，基于随机对照试验荟萃分析与观察性研究荟萃分析得出的干预措施不良反应风险估计值，整体上并无显著差异。这提示不良反应系统评价不应局限于特定的研究类型。详见本文后文的编辑摘要。