- Assessment Principles
- Search Strategy
- Assessment Process
- External Review Process
- Completed DAWG Reviews
The goal of the Drug Assessment Working Group (DAWG) of the Therapeutics Initiative is to clarify the state of scientific evidence regarding effectiveness and safety of drug therapy and to relate that evidence to the care of individual patients.
The DAWG consists of salaried employees (with expertise in systematic review methodology, clinical pharmacology, and epidemiology), Clinical Fellows in pharmacology, and graduate students from a range of medical disciplines. The DAWG systematically reviews and, when appropriate, critically appraises research relevant to new and existing drugs. The purpose of this document is to describe this systematic review process.
The DAWG follows a predefined protocol consisting of a research question, a literature search strategy, and a hierarchy of clinically validated outcome measures.
The research question asks whether the new drug provides a therapeutic advantage over similar drug therapy for a clinical condition (‘indication’) or versus placebo, in the case of the first drug therapy for a clinical condition. The DAWG relies on its own and external clinical experts both to define clinical conditions and to determine similar drug therapy.
The same DAWG members review all drugs. That is, the Working Group does not change as drugs change. And, all of the Working Group are expected to contribute to the design and conclusions of each systematic review, although individual members or groups take the lead for each project.
None of the DAWG members receives any funding directly or indirectly from the drug industry, nor do they or their families have any stock in any of these companies, except through unassigned mutual funds found in University pensions.
The Drug Assessment Working Group (DAWG) is committed to analyze scientific evidence on the effectiveness and safety of drug therapies used in Canada. They systematically review and, when appropriate, critically appraise research relevant to new and existing drugs. Often the goal is to discover whether a new drug provides a therapeutic advantage over existing similar drug therapy for a clinical condition. The Therapeutics Initiative has a dedicated team of researchers that is committed to the highest standards of research.
The drug assessment is guided by the following principles:
- Therapeutic advantage will be determined by utilizing a hierarchical framework. A hierarchical framework means that an assessment proceeds if, and only if, a series of conditions are met. If conditions are not met, then the assessment stops at that point. Therefore, the ordering of the hierarchy is crucial. The DAWG hierarchy:
- Type of study: The study must use the strongest possible design: almost always, the double and triple blind, randomized controlled trial.
- Type of participants: The study must include appropriate participants with an appropriate spectrum of disease from mild to severe; (i.e. a patient population found under normal conditions of clinical care).
- Type of intervention: The study must compare the new drug with an established drug with a similar mechanism of action; or with placebo, for drugs with a new mechanism of action. When the mechanism of action is unknown, comparators must have a similar clinical endpoint.
- Type of outcome measures: The study must use the most clinically valid outcome measures. If possible or practical, this should include final outcomes such as serious morbidity and mortality measures. Intermediate outcomes will be examined, but given less weight because they do not necessarily reflect ultimate morbidity or mortality.
- Generally, drug manufacturers have the obligation of demonstrating that their product provides therapeutic advantage over established therapies.
A predefined search strategy sets minimum inclusion and exclusion criteria based on major study design features listed above, such as patient selection and assignment, therapies chosen, appropriateness of outcome measures, and the degree of ‘blinding’ as to treatment allocation.
The search strategy only includes full clinical trial reports. Abstracts and poster presentations summarizing clinical trials are insufficient substitutes for full trial reports. Abstracts and poster presentations provide an outline of research design and a summary of major findings. However, they do not provide sufficient details to determine the validity/quality of the study or its findings. The latter requires analysis of how the trial was conducted and how the results were analysed.
The search includes trials submitted by the manufacturer and electronic data bases: MEDLINE 1966-present database, Current Contents 1996-present database, EMBASE 1988-present, and Cochrane database. In addition, retrieved clinical trials and some review articles are hand searched for references.
A ‘Search Findings’ section outlines all identified clinical trials and how they are categorized relative to the inclusion and exclusion criteria. Clinical trials and observational studies underway but not yet completed or published are also listed when submitted, but are not considered further in the systematic review unless full trial details are provided by the manufacturer.
The search strategy includes unpublished, full clinical trial reports submitted by the manufacturer to the Federal and Provincial regulatory and funding agencies, but considered proprietary. The TI considers this material in its systematic reviews. However, without publication, the Federal HPB, provincial Drug Benefit Plans, or review groups such as the TI make decisions regarding safety and efficacy, at least in part, on material that is not replicable or refutable. The TI believes that conducting and completing a clinical trial, but not submitting it for publication and providing full disclosure of the study report is unethical.
A full trial report in a peer reviewed journal is the preferred form of publication. Publication is sometimes delayed by journals. In these instances, copies of the articles as they will appear in publication are included in the drug assessment, along with proof of acceptance by the journal.
Another source of data are trials submitted for publication to a journal but refused publication. In this case the unpublished trial reports are given the full weight of a published trial, assuming reasonable proof of journal rejection is provided.
Clinical trial reports of trials comparing the correct treatments, using the best study design, and the most valid outcome measures are subjected to full critical appraisal. Critical appraisal means assessing the scientific validity of a study report following rigorous predefined criteria, most widely known through the Cochrane Collaboration publications. Critical appraisal methodology follows the work of Sackett et al,1 and is patterned after Schechter and LeBlanc.2
Each trial is appraised separately by at least two independent DAWG researchers. Discrepancies regarding assessments of the scientific validity of a clinical trial are resolved through consensus. Each drug assessment is discussed weekly by the full DAWG.
A concluding section of the report summarizes and draws conclusions from the body of available scientific evidence.
External Review Process
Approximately two-thirds of the DAWG reports are sent for external review. DAWG reports are not sent for external review if:
- the review is of a new formulation of an existing drug where the therapeutic advantage is limited to improved convenience and perhaps compliance with dosing regimen.
- the review is an update of a previously reviewed drug sent to reviewers, but no new published RCT evidence is found;
- the review is of a ‘me-too’ drug with an established mechanism of action and indication, and no valid published RCT evidence distinguishing the new drug from other members of a well established therapeutic category, such as beta-andrenergic blockers for hypertension or non-steroidal anti-inflammatory medications for osteo-arthritis.
For external review, the draft DAWG report, including the critical appraisal results, plus retrieved publications are sent to at least three experts (clinical or scientific) for assessment and comments. The reviewers are selected based on their knowledge of systematic review and critical appraisal methodology, their expert clinical or research knowledge of the disease studied, their ability to respond within a relatively short time-period (i.e. approximately 2 -3 weeks), and their willingness to declare any conflict of interest. Whenever possible, reviewers without conflicts of interest are chosen. Reviewers are asked, at minimum, to complete a checklist indicating the completeness of the review (any scientific evidence omitted) and whether they agree with the conclusions regarding critical appraisal.
The DAWG takes account of external reviews in the following ways:
1) New Citations are retrieved, and if they meet the inclusion criteria, are added to the review.
2) Internal Validity Issues. Of central concern to the DAWG, are any disagreements between it and external reviewers regarding the internal validity of the appraised clinical trials, particularly regarding key methodological steps such as randomization, the degree of intention to treat analysis, and clinical validity of outcome measures. These disagreements are discussed within the DAWG, and if the external reviewer’s comments are considered valid, the draft report is revised. If the DAWG does not consider the methodological issue raised by the external reviewer as valid, the external reviewer is contacted directly for discussion. If the issue remains un-resolved, both the DAWG and the external reviewer’s opinions are presented to the SIEC for consideration.
3) External Validity Issues. Interpreting external validity, meaning the extent to which research findings can be extrapolated to different contexts or individuals than those found in clinical trials, is not only difficult, but often the most difficult step in interpreting research findings, especially by individual clinicians. In part because of the inherent difficulty in establishing external validity, there are often disagreements between the DAWG and external reviewers regarding the interpretation of a therapeutic effect. The DAWG tends to be more conservative, while clinical reviewers are often more liberal in their degree of extrapolation to new individuals or contexts.
The DAWG recognizes that, although external validity issues are extremely important to individual clinicians, these concerns do not affect its mandate to establish the extent and strength of therapeutic effect in scientific evidence. Therefore, disagreements regarding external validity are not discussed directly with the external reviewers. Instead, both the DAWG and external reviewer’s opinions are presented to the Scientific Information and Education Committee (SIEC) for discussion.
Completed DAWG reviews
The reports from the three experts are presented to the SIEC before final conclusions are made. An attempt is made to reach a consensus. The conclusions are voted on and, if passed by a simple majority, are accepted as the position of the TI regarding the scientific evidence. Individual members of the SIEC, who are in a conflict of interest position on a particular drug, must declare this and absent themselves from the relevant discussion and vote.
- Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology: a basic science for clinical medicine. 2nd re.ed. Boston: Little, Brown and Company, 1991.
- Schechter M, and Leblanc FE. Critical appraisal of published literature. In: W. Hanstroidle, O. Spitzer, B. McPeek, D.S. Mulder, and M.F. Mckneally editors, Principles and practice of research: strategies for surgical investigators. New York: Springer-Verlag, 1986:112-17.
Dr. Tom Perry, MD, FRCPC
Tom graduated from McGill University Medical School in 1978. After a rotating internship at Dalhousie University and internal medicine residency in Vancouver, he achieved Fellowship in the Royal College of Physicians of Canada. He took additional training at the Karolinska Institute Department of Clinical Pharmacology in Stockholm 1986-87 and at UBC until 1989, when he was elected to the Legislative Assembly of British Columbia. Tom served as Opposition Health Critic from 1989-1991, then as Minister for Advanced Education, Training & Technology from 1991-93, and as a government MLA from 1993-96. These experiences alerted him to the importance of getting good value for money in health care, in order to maintain an effective universal health service in Canada.
After returning to clinical medicine in 1996, Tom practiced general internal medicine with sick patients at Delta Hospital, UBC Hospital and Vancouver General Hospital until 2014. Until the end of 2021, he maintained an outpatient internal medicine practice focused on pharmacological treatment of chronic pain and reducing polypharmacy (deprescribing). He continues to teach clinical pharmacology through seminars, lectures, special courses, and webinars presented throughout British Columbia. Tom has a special interest in the use of videography to teach students and health professionals about drugs and about human pathophysiology.
Tom co-chairs the Education Working Group and is Editor in Chief of the Therapeutics Letter. His other interests include wilderness canoeing and hiking, environmental conservation, peace and social justice issues, music, reading, and his wife (an experienced RN) and two children (geologist and NP). He likes continuous thinking and learning about medicine and drug therapy, and especially enjoys our interactions with smart and dedicated health care colleagues (students, MDs, pharmacists, NPs, nurses, PAs, and others) throughout B.C. and around the world. As of early 2022 he’s helping vaccinate British Columbians against Covid19 and hoping that the benefits of good medical science are extended rapidly to everyone on Earth.
Dr. Aaron M Tejani, BSc (Pharm), PharmD
Dr. Aaron M Tejani, is a researcher/educator with the Therapeutics Initiative (co-chair of the Education Working Group, member of the Drug Assessment Working Group), clinical assistant professor with the Faculty of Pharmaceutical Sciences (University of British Columbia), and Medication use evaluation pharmacist with Lower Mainland Pharmacy Services (Vancouver, BC). He completed his BSc(Pharm) at UBC (Vancouver) and Doctor of Pharmacy degree at Creighton University (Omaha, Nebraska).
Aaron is particularly interested in teaching healthcare professionals how to critically appraise evidence for medical interventions and how to use evidence in clinical practice/policy development. He is an author of a number of Therapeutics Letters.
Dr. Anat Fisher, MD, MHA, PhD
Dr. Anat Fisher has extensive experience using administrative health care databases to evaluate pharmaceutical policy changes and the long-term effects of medication. She also conducts systematic reviews of the efficacy and safety of new and established drugs. Anat has been a member of the Therapeutics Initiative since 2006, and is currently Co-Chair of the PharmacoEpidemiology Group (PEG). She completed her PhD in Pharmacology and Therapeutics with a focus on pharmacoepidemiology at the University of British Columbia. Her PhD thesis examined the comparative persistence of tumor necrosis factor alpha antagonists in patients with rheumatoid arthritis. Anat also holds an MD from the Hebrew University in Israel, and Master’s in Health Administration from the Tel Aviv University in Israel.
Dr. Anshula Ambasta, MD, MPH, FRCPC
Dr. Ambasta is a general internist with a research focus on healthcare quality and patient safety. Having completed a medical degree and post-graduate training in general internal medicine at the University of Calgary, Dr. Ambasta pursued a Masters in Public Health at the Harvard T.H. Chan School of Public Health with a focus on Clinical Effectiveness. She is an Assistant Professor in the Department of Anesthesiology, Pharmacology and Therapeutics at the University of British Columbia. Her overall research program focuses on reduction of low-value services in health systems. She is a member of the Choosing Wisely Canada national expert group dedicated to reducing unnecessary laboratory testing. Her research work in low-value laboratory testing has been funded by Alberta Health Services, Choosing Wisely Alberta, Canadian Society of Internal Medicine, Alberta Health Services, and Canadian Institutes of Health Research. Her ongoing research projects include implementation of a multi-modal intervention bundle to reduce low-value laboratory testing across hospitals in Alberta and British Columbia, collaboration with a patient and family advisory council to engage patients with reduction of low-value use of health care resources and describing linkages between low value use of diagnostic testing and therapeutic use in healthcare systems.
Ms. Aneesa Khan, BSc
Aneesa is a research assistant primarily working with Dr. Anshula Ambasta on the RePORT study, focused on patient engagement and reducing laboratory test overuse. She recently graduated from UBC with a BSc in Microbiology and Immunology, with a focus on topics in virology. Aneesa is enjoying working in health research while considering a future in medicine. Having worked for 3 years at the BC Emergency Medicine Network (UBC Department of Emergency Medicine) and on a number of health projects, her research interests span a variety of topics, including health equity, women’s health, and incorporating patient voices.
Dr. Wade Thompson, PharmD, MSc, PhD
Wade is a pharmacist and researcher working to ensure older persons are taking medications that are necessary, effective, safe, and consistent with their healthcare goals and treatment preferences. This primarily involves developing and evaluating strategies to stop medications when they are no longer a good fit (“deprescribing”). Wade approaches deprescribing and polypharmacy management research with a multi-methods approach, incorporating qualitative methods, pharmacoepidemiological methods, knowledge translation, and implementation science. He is also an investigator with the deprescribing.org initiative. Wade has worked clinically as a pharmacist in long-term care, geriatric outpatient clinics, and primary care clinics.
Mr. Ciprian Jauca, BA, DBM
Ciprian Jauca studied linguistics at the Babes-Bolyai University in Cluj-Napoca, Romania (major in Romance Languages, minor in Germanistic Studies) from 1986 to 1990. In 1991 he earned an International Diploma of Business Management and Intercultural Communication from the University of Osnabruck, Germany. He speaks several languages and has worked as translator and simultaneous interpreter, as well as managing social development projects for non-profit organizations. Ciprian joined the Therapeutics Initiative at its inception in 1994. He is the Managing Editor for the Therapeutics Letter and has been the Program Coordinator for the Therapeutics Initiative from its inception in 1994 until 2018. He created the Therapeutics Initiative website in the late 1990s and has been serving as its webmaster. He has been involved in the international Cochrane Collaboration since 2001, serving as the Managing Editor for Cochrane Hypertension. He has been involved in the International Society of Drug Bulletins since 2003, was elected as a member of the Executive Committee in 2008 and was elected Secretary General of the organization in 2016. Ciprian served as an elected member of the Board of Directors for the Association of Administrative and Professional Staff (AAPS) at the University of British Columbia from 2011 to 2015. Ciprian is currently serving as a member of the Board of Directors of the Laurier Institution, Treasurer and Chief Financial Officer of Colibri Learning Foundation and Vice-Chairman of the National Spiritual Assembly of the Bahá’í Community of Canada.
Dr. Benji Heran, PhD
Balraj (Benji) Heran received a B.Sc. (Hon.) in Physiology at the UBC and joined the TI in 2000. He recently graduated from the Ph.D. program in the Department of Anesthesiology, Pharmacology and Therapeutics, UBC. Under the supervision of Dr. Jim Wright, he conducted two systematic reviews of the dose-related blood pressure lowering efficacy of ACE inhibitors and angiotensin receptor blockers for primary hypertension. Benji is also a contributing author of a number of systematic reviews and protocols published in the Cochrane Library. From 2003 to 2009 he has served on the editorial team of the Cochrane Collaboration Hypertension Review Group as the Trial Search Co-ordinator, since 2009 has been an Editor with the Cochrane Hypertension Group and a Postdoctoral Research Fellow with the Cochrane Heart Group. He has a keen interest in cardiovascular research.
Mr. Stephen Adams, BSc
Dr. Carole Lunny, MPH, PhD
Dr Carole Lunny is a clinical epidemiologist and research methodologist with the Cochrane Hypertension Review Group, and the Therapeutics Initiative at the University of British Columbia. Dr Lunny specialises in methods for research synthesis and critical appraisal of systematic reviews, network meta-analyses, randomised controlled trials, overviews of reviews, and observational studies. She routinely tells people she is an expert in identifying bias and error in clinical research studies. Dr Lunny completed her PhD training at Cochrane Australia in January 2019 from Monash University’s School of Public Health and Preventive Medicine.
Dr Lunny’s current research is the development of a risk of bias tool for systematic reviews with network meta-analysis, tools for ‘overviews of reviews’, assessment of the impact of evidence syntheses on health systems and policy, and knowledge translation. She teaches courses at UBC and the University of Toronto.
To date, Dr Lunny has 24 publications, 16 of the 24 as first author, which have been cited 415 times as of July 2020. 16 out of the 24 studies are systematic reviews. You can access her publication at: https://pubmed.ncbi.nlm.nih.gov/?term=lunny+c&sort=date
Dr Lunny has been conducting systematic reviews since 2008 when she was hired as a Knowledge Synthesis Consultant with the International Centre for Infectious Diseases. She is an expert in conducting various types of knowledge syntheses including systematic reviews with and without meta-analysis, scoping reviews, evidence maps, network meta-analyses, meta-epidemiology and methods studies. She is an editor for the journal PeerJ and Systematic Reviews, and has peer reviewed for seven journals.
Dr Lunny is a member of the Cochrane Collaborations Statistical methods group and five other Cochrane methods groups, the SPOR Evidence Alliance, BC SUPPORT Unit, the Joanna Briggs Institute, the Campbell Collaboration, and the Open Science Taskforce at UBC.
Dr Lunny has been awarded over $515,000 in lifetime grant dollars and $257,000 in scholarship and award funding. She received two PhD scholarships– the International Postgraduate Research Scholarship and the Australian Postgraduate Award – from both the Cochrane Collaboration at Monash University (accepted) and the Joanna Briggs Institute (JBI) at University of Adelaide (declined). Dr Lunny worked for seven years for international development organisations such as the UNDP, UNICEF, the International Collaboration Centre for Infectious Diseases, the International Union Against Tuberculosis and Lung Disease in the US, Canada, Singapore, Myanmar, Thailand and Mexico.
Dr. Mohamed Ben-Eltriki, BSc (Pharm), RPh, MSc, PhD
Mohamed received his undergraduate pharmacy degree from the University of Benghazi, Libya in 2007 (Honours), completed his MSc in Pharmaceutical Sciences (Pharmacokinetics) from the University of Alberta in 2012, and his PhD in Experimental Medicine from the University of British Columbia in 2018. After his PhD, Mohamed joined the Therapeutics Initiative as a postdoctoral research fellow (Drug Assessment Working Group and Cochrane Hypertension Review Group), working under the join supervision of Prof. James Wright and Dr. Vijaya Musini. In addition, he maintains his patient care pharmacist license through front line work in community pharmacy. Dr. Ben-Eltriki’s current research focuses on clinical pharmacy, systematic review, evaluating drug safety and effectiveness, and critical appraisal of clinical practice guidelines.
You can access his publications at
Dr. Gloria Chu, BSc, PharmD
Gloria graduated with a Bachelor of Science (Honours Science) and Doctor of Pharmacy from the University of Waterloo in 2016. She is a community pharmacist working to incorporate deprescribing in a dispensary setting to try and minimize medication harm and burden. Gloria also enjoys getting to work with the Drug Assessment Working Group to learn critical appraisal skills, risk of bias assessments and work on systematic reviews.
Dr. Carolyn J Green, BHSc(PT), PhD
Carolyn’s research interests are organized around providing health care decision makers with research based as well as contextualized research. An active producer of health technology assessment (HTA) from 1992, she builds on a foundation of research synthesis methodologies, incorporating critical appraisal, meta-analysis, utilization analysis and decision analysis using data from administrative databases, systematic reviews and clinical trials.
Doctoral and postdoctoral training has added health informatics and qualitative research perspectives and approaches to her investigations into how knowledge is used in socio-technical systems. Carolyn has a BHSc(PT) from McMaster, a MSc from the Department of Health Care and Epidemiology at UBC, a PhD in Health Informatics from the University of Victoria, and has completed a CIHR sponsored postdoctoral fellowship in Knowledge Translation at the University of Alberta.
Dr. Josh Levin, MD, CM, CCFP, Dip. ABLM
I am a GP in Victoria BC with a special interest in lifestyle – specifically exercise as medicine – and the application of rational testing and prescribing. I graduated from McGill medicine in 2012, and obtained a diploma in lifestyle medicine from the American College of Lifestyle Medicine in 2019. In my free time I coach and compete at a Masters level in weightlifting, practice yoga and mindfulness, and love being out in nature!
Dr. Malcolm Maclure, PhD
Dr. Rita K McCracken, MD, PhD, CFPC(COE), FCFP
Rita McCracken is a full-service family doctor and researcher living as an uninvited visitor on the unceded, traditional territories of the Coast Salish peoples, including the Səl̓ílwətaʔ/Selilwitulh (Tsleil-Waututh), the xʷməθkwəy̓əm (Musqueam), and the Skwxwú7mesh (Squamish) Nations. She is a Assistant Professor in the Department of Family Practice at UBC and studies the availability of primary care in BC and reliable ways to measure changes in that availability. Her other research work includes assessing the effects of too much medication and effective methods to deprescribe those excessive medicines. She chose medicine as a second career after 10 years working in Human Resources, finished med school at the University of Calgary in 2006 and her doctoral studies at UBC in 2018.
Dr. Cait O’Sullivan, BA, BScPh, PharmD
Cait O’Sullivan has a clinical pharmacy background in acute general medicine, residential care, and community practice. She received a Bachelor of Arts from the University of New Brunswick (Honours, Medical Anthropology), a Bachelor of Science in Pharmacy from Dalhousie University, and a Doctor of Pharmacy Degree from the University of Washington. Cait has a research interest in the drug approval process and clinical practice guideline methodology.
Mr. Douglas M Salzwedel, BAA, MLIS
Douglas M Salzwedel obtained a Master of Library and Information Science from the University of Western Ontario in 2001 and has worked as the Information Specialist for the Therapeutics Initiative and Cochrane Hypertension since June 2009. In addition to information retrieval projects, he leads Cochrane Hypertension’s priority setting activities, and is a member of the Drug Assessment Working Group and the Communications and Outreach Group.
During his time with the TI, Douglas served on the Cochrane Information Specialists’ Executive from 2011 to 2017 and from 2015 to 2018 he provided induction training, software training, mentoring and ongoing support to Cochrane Information Specialists across Cochrane’s international network as a member of Cochrane’s Information Specialists’ Support Team. He re-joined the Cochrane Information Specialists’ Executive in 2023 and is also a Co-Convenor of PRESSforum, a search peer review portal for health sciences librarians engaged in systematic review searching
Prior to joining the Therapeutics Initiative, Douglas was the information specialist for the Cochrane Effective Practice and Organisation of Care (EPOC) Review Group and a librarian at the University of Ottawa’s Institute of Population Health.
Dr. Ken Bassett, MD, PhD
Ken Bassett conducts systematic reviews of the efficacy and safety of new and established drugs, as well as pharmaco-epidemiologic studies of serious adverse events associated with prescription drug therapy in British Columbia. His ongoing research interests are in the systematic review of drug therapy and drug funding policy.
Dr. James M. Wright, MD, PhD, FRCP(C)
James (Jim) Wright obtained his MD from the University of Alberta in 1968, his FRCP(C) in Internal Medicine in 1975 and his PhD in Pharmacology from McGill University in 1976. He worked as a specialist in Internal Medicine and Clinical Pharmacology from 1997-2021. He served as the Co-Managing Director of the Therapeutics Initiative and Editor-in-Chief of the Therapeutics Letter from 1994-2020. He currently sits on the Editorial Boards of PLoSOne and the Cochrane Library.
Dr. Wright’s research focuses on issues related to appropriate use of prescription drugs (particularly antihypertensive and lipid lowering drugs), Clinical Pharmacology, clinical trials, systematic review, meta-analysis and knowledge translation.