1 / 25

Indirect and mixed treatment comparisons

Indirect and mixed treatment comparisons. Hannah Buckley Co-authors: Hannah Ainsworth, Clare Heaps, Catherine Hewitt, Laura Jefferson, Natasha Mitchell, Carole Torgerson, David Torgerson. Overview. Exemplar trials Direct comparisons Indirect comparisons Methodological approaches

beadles
Download Presentation

Indirect and mixed treatment comparisons

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Indirect and mixed treatment comparisons Hannah Buckley Co-authors: Hannah Ainsworth, Clare Heaps, Catherine Hewitt, Laura Jefferson, Natasha Mitchell, Carole Torgerson, David Torgerson

  2. Overview • Exemplar trials • Direct comparisons • Indirect comparisons • Methodological approaches • Mixed treatment comparisons • Assumptions

  3. Exemplar trials • 2 RCTs of literacy interventions • EEF ‘writing bundle’ • Grammar for writing (GfW) 1 • Improving writing quality (IWQ) 2 • Improving writing quality of struggling year 6 pupils

  4. Grammar for writing • 15 guided writing sessions over 4 weeks • 53 schools • Progress in English 11 Long Form • Split plot design https://educationendowmentfoundation.org.uk/uploads/pdf/FINAL_EEF_Evaluation_Report_-_Grammar_for_Writing_-_February_2014.pdf

  5. Grammar for writing

  6. Grammar for writing

  7. Improving writing quality • Self-regulated strategy development combined with memorable experiences • 23 primary schools, 3 secondary school • Progress in English 11 Long Form • Cluster trial https://educationendowmentfoundation.org.uk/uploads/pdf/EEF_Evaluation_Report_-_Improving_Writing_Quality_-_May_2014.pdf

  8. Direct comparison • Treatment effect estimates usually from direct comparisons between two treatments in an RCT

  9. Direct comparisons MD= 0.78 95% CI: (0.00, 1.56) MD= 2.53 95% CI: (0.90, 4.16) MD = mean difference

  10. Indirect comparisons • Used to provide estimates when evidence from direct comparisons not available • Adjusted indirect comparison (IC) - common comparator required

  11. IC methods - overview • Frequentist/Bayesian approach • Naïve IC (no advantages of RCT) • Adjusted IC • Meta-regression • Generalised linear mixed models (IPD data) • Confidence profile method • Bayesian Markov chain Monte Carlo (MCMC)

  12. IC methods – simple adjusted • Frequentist approach • Uses aggregate trial data • Adjusted based on common comparison • Estimates from trials extracted • If more than one trial for each comparison then use a weighted combination as in meta-analysis

  13. IC methods –adjusted

  14. IC methods – meta-regression • Frequentist approach • Uses aggregate data • Fixed/random effects • ӨBC modelled as a function of one or more study characteristics as predictor variable(s) • Co-efficient of indicator for comparison gives effect estimate

  15. Indirect comparison • Comparing GfW and IWQ

  16. IC - example Effect estimate for B vs C: ӨBC = ӨAB –ӨAC = 2.53–0.78 = 1.75

  17. IC - example Variance of effect estimate for B vs C: var(ӨBC) = var(ӨAB)+var(ӨAC) = 0.69 + 0.16 = 0.85

  18. IC - example

  19. IC - example • No evidence of a difference in means between pupils receiving each intervention • with a non-significant increase of 1.75 marks (95% CI: -0.06, 3.56) in writing score for those receiving the IWQ intervention compared with those receiving the GfW intervention

  20. Assumptions – IC • Homogeneity assumption: • 2 test • I2 • Similarity assumption in terms of effect moderators • populations should be similar in both sets of trials • participants in trial AB could have been randomised in trial AC • Same estimate would be obtained in trial ABC

  21. Mixed treatment comparison • Combining direct and indirect evidence • Indirect evidence supplements direct evidence • 1 RCT of direct evidence is as precise as indirect evidence based on 4 RCTs 3

  22. Assumptions - MTC • Consistency • indirect estimate would be the same as estimate from direct evidence

  23. Conclusions • Indirect comparisons can provide of relative effectiveness • MTC may provide gains in precision • Methods may be particularly applicable in an education setting where BAU frequently used as a comparator • Caution must be taken with interpretation

  24. References • Torgerson, D., Torgerson, C., Mitchell, N., Buckley, H., Ainsworth, H., et al. (2014). Grammar for Writing Evaluation Report and Executive Summary. Published by the Education Endowment Foundation on educationendowmentfoundation.org.uk. Last accessed 09 Sep 2015. • Torgerson, D., Torgerson, C., Ainsworth, H., Buckley, H., Heaps, C., et al. (2014). Improving Writing Quality Evaluation Report and Executive Summary. Published by the Education Endowment Foundation on educationendowmentfoundation.org.uk. Last accessed 09 Sep 2015. • Glenny, A, D Altman, et al. (2005) Indirect Comparisons of Competing Interventions. Health Technology Assessment vol. 9, no. 26.

  25. Resources • Bucher, Heiner C. et al. (1997) The Results of Direct and Indirect Treatment Comparisons in Meta-Analysis of Randomized Controlled Trials. Journal of Clinical Epidemiology, vol. 50, no. 6: 683–91. • Miladinovic, B., et al (2014). Indirect Treatment Comparison. Stata Journal vol 14, no. 1: 76–86. • Jansen JP, et al (2011). Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. Value in Health vol 14, no. 4: 417-28.

More Related