Introduction to a new risk of bias tool for network meta-analysis

Introduction to a new risk of bias tool for network meta-analysis

This live webinar has already taken place. Scroll down to view the video recording.

Dr. Carole Lunny described a new tool designed specifically to assess the risk of bias in network meta-analyses. Network meta-analyses (NMA) provide comparative evidence about the effects of all healthcare interventions for a given condition. However, NMA results can be undermined by flaws in design, conduct, analyses and reporting. Several tools are available for the critical appraisal of an NMA. However, none aim specifically to assess the risk of bias in a network meta-analysis.


TITLE: Introduction to a new risk of bias tool for network meta-analysis

WHEN: Wednesday, September 27th, 2023 at 12:00 PM (noon) PDT [convert to your local time]

WHERE: This is a free online webinar.

SPEAKER: Dr. Carole Lunny, Research Associate, Knowledge Translation (KT) Program, University of Toronto (UofT)

About the topic: A new tool was designed specifically to assess the risk of bias in network meta-analyses was developed using rigorous methodology by an international steering committee. The committee developed a protocol and made conceptual decisions about the tool’s structure, developed the tool using a multi-step approach: definition of the scope of the tool; conduct a methodological review to compile a preliminary list of concepts related to bias in NMAs; web-based Delphi survey to collate expert opinion regarding the quality of NMAs; online survey to solicit decision makers needs; and refinement through piloting.

This new RoB NMA tool includes three domains: bias arising from the network characteristics/geometry, bias arising from potential effect modifiers, and bias arising from statistical synthesis.  The three domains comprise a total of 17 statements which can identify potential limitations of the NMA, leading to a judgement about the risk of bias within each domain (low, high or unclear risk of bias).

The tool should be used to identify potential limitations in the way an NMA was conducted that may lead to bias in its results or conclusions. The RoB NMA tool should be used as a supplement to the assessment of the risk of bias in the underlying evidence review, for example using the ROBIS tool. Piloting of the RoB NMA tool is currently ongoing (anyone interested in piloting the tool please reach out). The team is confident that the new RoB NMA tool will allow objective rating of the risk of bias in an NMA.

About the speaker: Dr. Carole Lunny is a methodologist in evidence synthesis, and Research Associate at the Knowledge Translation (KT) Program, University of Toronto (UofT). She completed two postdocs: one with the Cochrane Hypertension Group at the University of British Columbia (UBC) and the second with the Knowledge Translation (KT) Program at the University of Toronto. Dr. Lunny received her PhD as a Clinical Epidemiologist and Methodologist with Cochrane Australia at Monash University in Melbourne, Australia. Working with these organisations has refined her skills in knowledge translation, machine learning, network meta-analysis (NMA), development of critical appraisal tools, and grant writing. Dr. Lunny has published 52 peer-reviewed publications: 27 as first author and 4 as senior author. She has been awarded $3.8 million in lifetime grant dollars and $260,000 in scholarship and award funding. Dr. Lunny has established research focused on the development of methods and tools for evidence synthesis, to help researchers, practitioners, and policymakers make evidence-informed decisions about which interventions are best (namely the RoB NMA tool and WISEST AI tool). However, she is most widely known internationally for her seminal research into ‘overviews of systematic reviews’ (i.e. umbrella reviews, meta-reviews). Her full research works can be accessed at

About the TI Methods Speaker Series: The TI Methods Speaker Series are offered free of charge and everyone is welcome. The event is held at noon on the last Wednesday of each month via videoconference. The presentations are recorded and the video recordings are posted online. Click here to view a list of talks offered in 2023.

No Comments

Post A Comment