27 Oct TI Methods Speaker Series: Impact of reporting guidelines and prevalence of inconsistency in systematic reviews with network meta-analysis
What is the impact of PRISMA-NMA in published NMAs? Which items require attention and improvement? What is the prevalence of inconsistency, and its association with different NMA characteristics? NMAs published after 2015 more frequently reported the five NMA items. However, improvement in reporting after 2015 is similar to that prior to 2015, suggesting that PRISMA-NMA is not the only factor accelerating this improvement. Inconsistency was more common than what would be expected by chance, suggesting that researchers should devote more resources to exploring how to mitigate inconsistency.
This webinar has already taken place. Please scroll down to view a video recording of the event.
WHEN: Wednesday, October 27th, 2021 from 12:00 to 1:00 PM PDT [click here to convert to your local time]
WHERE: Offered online using the Zoom platform.
TITLE: Impact of reporting guidelines and prevalence of inconsistency in systematic reviews with network meta-analysis
SPEAKERS: Areti-Angeliki Veroniki (Scientist at the Li Ka Shing Knowledge Institute of St. Michael’s Hospital, Unity Health Toronto, Canada) and Sofia Tsokani (PhD Student and Research Associate at the University of Ioannina, Greece).
Background: Numerous, competing interventions are available for any given condition. Network meta-analysis (NMA) has attracted growing interest in evidence-based medicine, since it provides the comparative effectiveness and potential for harm across a network of interventions. In 2015, the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses for network meta-analysis) reporting guideline was extended for NMA, and comprised 32 items (27 core items for systematic reviews plus 5 items for NMAs). A key reporting item in PRISMA-NMA is consistency between direct and indirect evidence in a network of interventions. The assumption of consistency is fundamental to the validity of the NMA results.
Objective: To indicate the impact of PRISMA-NMA in published NMAs, highlight which items require attention and improvement, estimate the prevalence of inconsistency, and describe its association with different NMA characteristics.
Methods: We searched MEDLINE, EMBASE and the Cochrane Database of Systematic Reviews from inception until July 2018 for NMAs of randomized controlled trials (RCTs). We assessed the NMA’s completeness of reporting, and compared the average reporting score of NMAs published before and after 2015. We used the original 32-point scale PRISMA-NMA and a modified 49-point scale evaluating all multiple-content items. In networks with at least one closed loop, we assessed inconsistency using the design-by-treatment interaction (DBT) model and estimated the prevalence of inconsistency. We explored the association between inconsistency and network characteristics (e.g., number of studies and treatments) and heterogeneity.
Results: We included 1,144 NMAs, where the mean modified PRISMA-NMA score was 32.1 (95% CI 31.8-32.4). For each year, the mean modified score increased by 0.96 (95% CI 0.32-1.59) for 389 NMAs published before 2015 and by 0.53 (95% CI 0.02-1.04) for 755 NMAs published after 2015. Adjusting for journal impact factor, type of review, funding, and treatment category, the mean modified PRISMA-NMA score for NMAs published after 2015 was higher by 0.81 (95% CI 0.23-1.39). Reporting of summary effect sizes to be used, individual study data, sources of funding for the systematic review and role of funders decreased after 2015 by 6-16%. Inconsistency was prevalent in 14% of the networks, whereas networks comparing few interventions in many studies were more likely to have small DBT p-values. In the presence of inconsistency, the consistency model displayed higher heterogeneity than the DBT model.
Conclusions: NMAs published after 2015 more frequently reported the five NMA items. However, improvement in reporting after 2015 is similar to that prior to 2015, suggesting that PRISMA-NMA is not the only factor accelerating this improvement. Also, inconsistency was more common than what would be expected by chance, suggesting that researchers should devote more resources to exploring how to mitigate inconsistency.
About the speakers:
- Dr. Areti-Angeliki Veroniki is a Scientist at the Li Ka Shing Knowledge Institute of St. Michael’s Hospital, Unity Health Toronto, Ontario, Canada, and a co-convenor of the Cochrane Statistical Methods Group. She is a mathematician, holds an MSc in Statistics, and a PhD in epidemiology and biostatistics. Her expertise lies in meta-epidemiology and network meta-analysis.
- Ms. Sofia Tsokani is PhD student and Research Associate in the Department of Primary Education, School of Education, in the University of Ioannina, Greece. She holds BSc in Mathematics and a MSc in Statistics and Operations Research. Her research interests focus on systematic reviews, and statistical modelling for meta-analysis and network meta-analysis of interventions and diagnostic tests.
About the TI Methods Speaker Series: The TI Methods Speaker Series are offered free of charge and everyone is welcome. The event is held at noon on the last Wednesday of each month. During the COVID-19 pandemic, while physical distancing measures are in effect, the TI Methods Speaker Series are offered via videoconference. The presentations are recorded and the video recordings are posted online. Click here to view the scheduled topics for 2021 and click here to view a list of TI Methods Speaker Series talks offered in 2020.