Professor Irina Volkov, Director of Evidence-Based Medicine at a teaching hospital, faces a complex challenge for her upcoming presentation to the National Institute for Health and Care Excellence (NICE). She's been asked to provide expert testimony about treatment options for a chronic condition where multiple systematic reviews exist, but they reach conflicting conclusions. Her analysis reveals three Cochrane systematic reviews (published 2019-2023), two rapid reviews commissioned by different health technology assessment agencies, four network meta-analyses comparing different intervention combinations, and two umbrella reviews attempting to synthesise the systematic reviews. The reviews use different inclusion criteria, quality assessment methods, and statistical approaches. Some conclude there is strong evidence for intervention A, others favour intervention B, and several suggest insufficient evidence for any recommendation. The NICE committee needs Irina's expert interpretation to guide national treatment guidelines that will affect prescribing decisions across the NHS, potentially impacting 250,000 patients annually and £50 million in healthcare spending. 1. I'd like the group to develop a systematic approach for Irina to evaluate the quality and reliability of these conflicting reviews. Please discuss how tools like AMSTAR-2 should be applied, and bring evidence about which quality domains are most critical when reviews reach different conclusions about the same intervention. 2. Given the mix of review types Irina is dealing with, I want the group to critically analyse how different synthesis methodologies should be weighted in her evidence evaluation. Please discuss the relative strengths of Cochrane reviews versus network meta-analyses versus umbrella reviews and how methodological rigor should influence evidence interpretation. 3. Looking at the conflicting conclusions, I'd like the group to explore the potential sources of discordance between reviews. Please discuss how differences in inclusion criteria, search strategies, quality assessment, and statistical methods could lead to different conclusions and how Irina should present this uncertainty to the NICE committee. 4. Expert Synthesis and Recommendations Finally, I want the group to consider Irina's role as an expert synthesiser of conflicting evidence. Please discuss how she should move beyond simply describing the reviews to providing informed interpretation that guides policy decisions. What frameworks exist for expert evidence synthesis, and how should uncertainty be communicated to decision-makers? The following two articles could help you: Shea, B.J., Reeves, B.C., Wells, G., Thuku, M., Hamel, C., Moran, J., Ivory, J., Tugwell, P., Welch, V. and Kristjansson, E. (2017) 'AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions', BMJ, 358, j4008. Available at: https://doi.org/10.1136/bmj.j4008 Salanti, G. (2012) 'Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, one statistical framework', Research Synthesis Methods, 3(2), pp. 80-97. Available at: https://doi.org/10.1002/jrsm.1037 Ahmed
AMSTAR-2 critical domains systematic review quality assessment network meta-analysis methodology
AMSTAR-2 systematic review quality appraisal critical domains
conflicting systematic reviews discordance inclusion criteria heterogeneity
network meta-analysis indirect comparison evidence synthesis methodology
AMSTAR-2 16 domains critical items systematic review quality Shea 2017 BMJ
AMSTAR-2 appraisal systematic reviews confidence overall rating
umbrella review methodology evidence synthesis overlapping systematic reviews
GRADE evidence certainty health technology assessment guideline uncertainty
AMSTAR-2 critical domains seven items systematic review confidence rating
Salanti network meta-analysis transitivity consistency assumption indirect comparisons
sources of discordance between systematic reviews conflicting conclusions inclusion criteria search strategy heterogeneity