Systematic Review

• Follows a strict, reproducible protocol (PRISMA guidelines)
• Includes a comprehensive literature search across multiple databases
• Uses pre-defined inclusion and exclusion criteria
• Assesses the quality and risk of bias of each included study
• May or may not include a quantitative synthesis (meta-analysis)

Interpretation of the Forest Plot

Structure of the Forest Plot

Study-by-Study Data

Overall (Pooled) Result

• Overall HR = 1.50 (95% CI: 1.14 - 1.98) - shown by the diamond
• Since the entire diamond lies to the right of 1.0, this is statistically significant
• Interpretation: Pioglitazone is associated with a 50% increased risk of bladder cancer compared to non-pioglitazone users

Heterogeneity

• I-squared (I²) = 78.8% - this is high (>75%), indicating substantial heterogeneity between studies (the studies do not all agree with each other)
• p-value = 0.000 - the heterogeneity is statistically significant
• Because of high heterogeneity, a random effects model was used (noted at the bottom of the plot), which is appropriate as it accounts for between-study variability

X-axis Direction

• Left of 1 (toward 0.1) = Favours Pioglitazone (pioglitazone is protective/reduces risk)
• Right of 1 (toward 10) = Favours non-pioglitazone (pioglitazone increases risk)
• The overall result falls to the right of 1, confirming increased bladder cancer risk with pioglitazone

Conclusion from the Forest Plot

How to Interpret a Forest Plot

1. The Basic Anatomy

Study Name    |----■----|        HR (95% CI)     Weight
Study Name      |--■--|          HR (95% CI)     Weight
Study Name    |--------■-----|   HR (95% CI)     Weight
              ◆                  Pooled estimate
         _____|_____
        0.1   1    10
     Favours A   Favours B

2. Reading Statistical Significance

For individual studies:

• If the horizontal CI line crosses the vertical null line (value 1) → NOT statistically significant
• If the entire CI line is on one side of the null line → Statistically significant

For the overall pooled result:

• If the diamond touches or crosses the null line → pooled result is NOT significant
• If the diamond lies completely on one side → pooled result IS significant

3. Direction of Effect

Always check the axis labels - they are not always the same in every forest plot.

4. Effect Size Measures Used

5. Study Weight

• Each study is assigned a weight (shown as % or by box size)
• Weight is determined by sample size and precision (narrower CI = higher weight)
• Studies with larger sample sizes and smaller variance contribute more to the pooled estimate
• In fixed-effects models: weight = inverse of within-study variance
• In random-effects models: weight = inverse of total variance (within + between study)

6. Heterogeneity - The Most Important Concept

I² Statistic (most commonly used):

Cochran's Q test (p-value for heterogeneity):

• p < 0.10 is usually taken as significant heterogeneity (note: threshold is 0.10, not 0.05)

What to do when heterogeneity is high:

• Use a random-effects model instead of fixed-effects
• Perform subgroup analyses to find the source
• Consider whether pooling is appropriate at all

7. Fixed-Effects vs Random-Effects Model

8. Step-by-Step Approach in an Exam

1. Read the title - what is the exposure/intervention and outcome?
2. Check the x-axis - what effect measure is used? What do the two sides mean?
3. Look at each study - does the CI cross 1? Which way does the point estimate lean?
4. Check box sizes/weights - which studies are driving the result?
5. Look at the diamond - does it cross the null line? What is the pooled estimate and its CI?
6. Check I² and p-value - is there heterogeneity? Was random or fixed effects used?
7. State the conclusion - is the overall result significant? In which direction? With what caveats?

9. Applied to the Pioglitazone Forest Plot (from Question 3)

1. Topic: Pioglitazone vs no pioglitazone; outcome = bladder cancer (HR used)
2. X-axis: Left = favours pioglitazone; Right = favours non-pioglitazone (more cancer risk)
3. Individual studies: Most CIs cross 1 (not individually significant) except Piccinni 2011 (HR 4.30) and Neumann 2012 (HR 1.22)
4. Weights: Neumann 2012 (16.96%) and Chang 2012 (15.48%) carry the most weight; Song 2012 (1.61%) has the least
5. Diamond: Sits entirely to the right of 1 - HR 1.50 (1.14-1.98) - statistically significant
6. Heterogeneity: I² = 78.8%, p = 0.000 - high heterogeneity - random-effects model was correctly used
7. Conclusion: Pioglitazone is associated with a 50% increased risk of bladder cancer, though high heterogeneity means this pooled estimate should be interpreted cautiously

Heterogeneity in Meta-Analysis

Definition

Types of Heterogeneity

1. Clinical Heterogeneity

• Example: Studies comparing the same drug but in different age groups, doses, or disease severity
• Cannot be measured statistically - must be judged by the reviewer

2. Methodological Heterogeneity

• Example: Some studies are RCTs, others are cohort studies; different follow-up durations; different blinding methods

3. Statistical Heterogeneity

• This is the type that is formally tested and reported in a forest plot
• Results from clinical + methodological heterogeneity

How Heterogeneity is Measured

A. Cochran's Q Test

• Calculates the weighted sum of squared differences between each study's result and the pooled result
• Tests the null hypothesis: "All studies share a common true effect"
• p < 0.10 (not 0.05) is used as the threshold for significant heterogeneity
• Limitation: Has low statistical power with few studies - may miss true heterogeneity when only 3-4 studies are included

B. I² Statistic (Most Widely Used)

I² = [(Q - df) / Q] × 100% where Q = Cochran's Q statistic, df = degrees of freedom (number of studies - 1)

C. Tau² (τ²)

• Estimates the between-study variance (used in random-effects models)
• Tau = τ = the standard deviation of true effects across studies
• Larger τ² = more spread in the true effect sizes across studies
• Reported alongside I² in modern meta-analyses

Sources of Heterogeneity

What to Do When Heterogeneity is High

1. Use a Random-Effects Model

• Assumes the true effect varies across studies (rather than being the same)
• Accounts for both within-study and between-study variance
• Gives wider, more conservative confidence intervals
• Preferred when I² > 50% (and mandatory when I² > 75%)

2. Subgroup Analysis

• Divide studies into clinically meaningful subgroups and pool within each group
• Example: pool RCTs separately from observational studies; pool by patient age group

3. Meta-Regression

• Statistical technique to explore whether a continuous variable (e.g., mean age, dose, year of study) explains the variation in effect sizes

4. Sensitivity Analysis

• Re-run the analysis excluding outlier studies to see if they are the source of heterogeneity
• If removing one study dramatically changes I², that study was likely driving the heterogeneity

5. Narrative Synthesis

• When heterogeneity is too high to justify pooling, describe findings qualitatively rather than producing a combined estimate

Impact on the Forest Plot

Applied Example (from the Pioglitazone Plot)

• I² = 78.8%, p = 0.000 → High heterogeneity
• Looking at the plot: Piccinni 2011 has an HR of 4.30, while Chang 2012 has only 1.06 - these diverge widely
• Action taken: Random-effects model was correctly used
• Implication: The pooled HR of 1.50 must be interpreted cautiously because the studies do not agree well with each other

Key Rule to Remember

High I² does not invalidate a meta-analysis - it simply means the pooled estimate should be interpreted with more caution, and the sources of variation should be explored. A meta-analysis with high heterogeneity but a well-reasoned random-effects model is still scientifically valid.