Understanding and doing evidence based research in Pathology

1. Understanding and doing evidence based research in Pathology I. The Question Randomized Controlled Trials (with thanks to Brian Haynes) Observational Trials (with thanks to Mitchell Levine)

2. Goals of presentation • Describe study types • Descriptive • Analytic • Discuss how to formulate a research question • Discuss the study designs that will answer the research question

3. Making evidence based decisions • Making decisions • What is this disease? • What additional tests should be done? • How will this disease behave? • What information do I need to provide for management purposes? • To make evidence based decisions, the research informing your decisions must provide the closest possible approximation to the truth

4. Research types (“clinical” research”) • Case report • Descriptive studies (Observational) • Case series • Analytic studies • measure associations • comparison group present • case control, cohort (Observational) • RCT (clinical trial)

5. The question • PICO (T) • Population • Intervention • Comparison intervention • Outcome • (Time) • Structures the question

6. RCT question • In patients with clinically and histologically node negative breast cancer, does sentinel node biopsy alone achieve the same survival as axillary lymph node dissection? • P = patients with clinically and histologically node negative breast cancer • I = sentinel node biopsy • C = axillary lymph node dissection • O = survival NSABP B-32. Krag et al. Lancet Oncology 2010;11:927-33

7. RCT question • In patients with metastatic squamous cell carcinoma to cervical lymph nodes diagnosed on FNA, what proportion have positive test results when the FNA is tested for oncogenic HPV-DNA, E6 oncoprotein overexpression or p16/Ki-67 dual staining cytology with HPV triage? (A. Abdel-Mesih) • P = patients with metastatic squamous cell carcinoma to cervical lymph nodes diagnosed on FNA • I1 = E6 oncoprotein overexpression • I2 = p16/Ki-67 dual staining cytology with HPV triage • C = oncogenic HPV-DNA test • O = positive test

8. Observational question • PICO • Replace “intervention” by “exposure”

9. Observational study (single cohort/ “case series”) • Prognosis • What is the 5 year survival in women with breast cancer who have axillary lymph node metastases no larger than micrometastases or ITC’s? • P = women with breast cancer who have axillary metastases no larger than micrometastases or ITC’s • I/E = time • O = survival • T= 5 years Or • P = women with breast cancer • I/E = axillary metastases no larger than micrometastases or ITC’s • O = survival • T = 5 years

10. What is the 5 year survival in women with breast cancer who have axillary lymph node metastases no larger than micrometastases or ITC’s? • Here there is no comparison group. But maybe what the patient really wants to know (patient centred or patient relevant outcome) is whether her survival is adversely affected by the presence of micromets or ITC’s compared to what her survival would be if she had negative nodes. You can infer this by seeing what the literature says in regards to node negative survival, or you could do a direct comparison

11. Observational study (cohort) with comparator group • In women with breast cancer, do micrometastases or ITC’s in axillary lymph nodes decrease 5 year survival as compared to women with axillary node negative breast cancer? • P = women with breast cancer • I / E = axillary node micromets/ITC • C = negative axillary nodes • O = survival • T= 5 years • Here you have 2 groups • Group 1 survival • Group 2 survival

12. Observational study (cohort) with comparator group • Subgroup of DLBL, now called BCLU • Is BCLU a distinct entity other than on histology? • Plan was to look at clinical presentation and OS and DF survival. OS was the more important outcome, so it was made the primary outcome • Do patients with BCLU have decreased OS compared to patients with DLBL? • Among patients traditionally called DLBL , is BCLU histology associated with decreased overall survival compared to those with conventional DLBL histology? (L. Edwards) • P = patients traditionally diagnosed as DLBL • I / E = BCLU histology • C = DLBL histology • O = overall survival • Here again you have 2 groups

13. Observational study (cohort)with comparator group • In patients with endometrioid carcinoma of the endometrium, does LVSI confer decreased survival? • LVSI is a “prognostic” factor • you are still dealing with 2 groups, one with and one without the prognostic factor: Among patients with endometrioid carcinoma of the endometrium, is survival decreased in those with LVSI, as compared to those without LVSI? • P =patients with endometrioid carcinoma of the endometrium • I / E = LVSI • C = no LVSI • O = survival (OS, Disease Specific Survival?) • T = ?

14. Diagnostic testing (1) • In women with a papillary lesion on needle core biopsy, does using a diagnostic algorithm identify lesions that require surgical excision? • In women with a papillary lesion on NCB, what is the sensitivity and specificity of a histologic diagnostic algorithm for identifying ADH or worse lesion, as assessed from the surgical excision? (LJ Elavathil) • P = women with a papillary lesion on NCB • I = use of a diagnostic algorithm when reading the core (“histologic diagnostic algorithm”) • O = ADH or worse in the excision specimen

15. Diagnostic testing (2) • In women referred to colposcopy with ASCUS or LSIL on screening Pap smear, what is the accuracy of p16/Ki-67 dual stain cytology for detecting histologically confirmed CIN 2+? (M. Schell) • P = women referred to colposcopy with ASCUS or LSIL on screening Pap smear • I = p16/Ki-67 dual stain cytology • O = histologically confirmed CIN 2+

16. Studies of association • The preceding questions asked whether the intervention or exposure was associated with the outcome • There is hierarchy of study design • RCT gives the most credible association of intervention with outcome • RCT cannot be done for • exposures that cannot be assigned, e.g. LVSI in tumors • studies of harm e.g. give DES to pregnant women to confirm suspected adverse effect on fetus • RCT may not be feasible if outcome is rare or requires prolonged time to occur • Designs that are more prone to bias may need to be used

17. Randomized controlled trials • You create the intervention • Randomization (if concealed) ensures groups differ only in the intervention • Both unknown and unmeasured important prognostic factors are likely to be similar among groups because of random allocation • Only way to control for unknown prognostic factors that can influence the outcome of interest

18. Controlling for factors that influence outcome rather than do RCT • Controlling for known factors (create subgroups or do statistical adjustment) • In women breast cancer, do micrometastases or ITC’s in axillary lymph nodes decrease 5 year survival as compared to women with axillary node negative breast cancer? • Tumor size, LVSI, hormone receptor status, treatment • In patients with endometrioid carcinoma of the endometrium, does LVSI confer decreased survival? • Stage • FNA HPV test positivity • Sequence of testing and sample adequacy • What about unknown factors?

19. Evidence suggesting results from RCTs are more valid than from observational studies • Example: vitamin E • cohort study (n >5000) showed significantly fewer deaths due to CAD • RCT (n>5000) showed no effect , and harm from high doses • Cochrane review of observational studies showed frequently worse prognosis (i.e. sicker patients) at entry among control patients • The result is biased from the beginning; intervention will appear more successful even if it is not

20. Randomized controlled trials • Are they relevant to pathology? • Will you ever be the PI or design a RCT? • Yes, why not if the research question is relevant to you? • HPV testing for cervical cancer screening: you read the Pap smears, you have the right to evaluate the alternatives • Will you be a co-investigator? • Will you need to interpret the published study to decide to change or maintain current service provision?

21. Why start with RCT? • Concepts in RCT’s applicable to all study designs • Concepts are easiest to understand in the RCT design

22. RCT: Itemized checklist • CONSORT statement • Parallel group randomized trials • Extension for trials assessing nonpharmacologic treatments

23. Design issues • Defining the population • Defining the intervention • Generating randomization sequence • Defining the outcome (s) • Blinding • Sample size • ALL except randomization are pertinent to other study designs as well

24. Sentinel node vs axillary dissection in breast cancer NSABP B-32Krag et al. Lancet Oncology 2010;11:927-33 • Population • Inclusion criteria • Women with invasive breast cancer • Clinically negative nodes • Sentinel nodes negative on HE (and negative IHC when performed for confirmation of suspicious findings on HE) • Exclusion criteria

25. NSABP B-32 Women with breast cancer and clinically negative axillary nodes • Stratification • - Age ≤49, ≥50 • cT ≤2, 2.1-4.0, ≥4.1 cm • Surgery (lumpectomy, mastectomy) R SLN only SLN + ALND SLN negative SLN negative • Follow to outcome

26. InterventionNSABP B-32 • Intervention: sentinel node dissection, lymph nodes sectioned at 2mm, no routine IHC • Control: SLN dissection with ALND; and SN sectioned at 2mm, no routine IHC

27. Randomization and Intervention Assignment • Want every participant to have equal chance of being assigned to intervention or control, regardless of clinical status • Random assignment to group • Generate unpredictable allocation sequence • So that no one can predict what is the allocation assignment

28. Simple Randomisation Unequal sample sizes if study small (<200) Unequal patients with important prognosticators: stratify

29. Randomization • Generate unpredictable allocation sequence • Conceal this sequence until assignment • Call in to study centre √ • Sealed envelopes

30. Intervention and comparison • Should be indistinguishable in appearance, taste, etc • Easier to do in drug trial • May not be possible in surgical trial

31. Outcome • NSABP B-32 • primary outcome: overall survival • Secondary outcomes: DFS, second cancers, etc • Important to specify • Sample size must be sufficient to detect at least the primary outcome • Secondary outcomes more susceptible to false positive results (issue of p value and multiple comparisons)

32. Defining the Outcome • What is the outcome? • Patient important outcomes • Survival: OS? Disease specific? • How will you measure or identify the outcome? • Survival: clinic charts? Contact family doc/family? Death registry? • Adjudication for subjective outcomes • e.g. CIN

33. Outcome • Composite outcomes • e.g. CV death, stroke, MI, angina • Useful for increasing events, therefore decreasing sample size • UKPDS: 3867 DM2 x 10y (L.A. Moye. Multiple analyses in clinical trials. Springer, NY, 2003) • diabetes-related endpoints: sudden death, death from hypo/hyperglycemia, fatal /non-fatal MI, angina, heart failure, stroke, renal failure, amputation, vitreous hemorrhage, retinal photocoagulation, blindness in 1 eye, cataract • Intensive vs control Tx : 35.3% vs 38.5%, p=0.029 • Difference 3.2% (retinal photocoagulation 2.7%)

34. Surrogate vs major outcomes • Outcomes that are not themselves important to patients, but are associated with outcome that ARE important to patients • Bone density for fracture (not so good) • CIN 2+ for cervical invasive squamous carcinoma (OK)

35. Blinding • Can potentially blind • Participants, health care providers, data collectors, outcome adjudicators, data analysts • Should blind if knowing allocation can cause differential reporting and search for outcomes, especially if outcomes are subjective • May not be needed for objective outcomes such as death • Prevent co-intervention, contamination

36. Sample size • Small studies • Significant differences: the 2 groups may have different prognoses and therefore differences may be due to chance • If events are few and no significant difference, cannot exclude possibility of important difference • Need adequate sample size • Too small, problems as above • Too large, waste resources, potentially put patients at risk • Can calculate estimated sample size • Need to know baseline risk in control group, the clinically important difference one want s to see, required power, alpha

37. SAMPLE SIZE NSABP B-32 • Trial was powered to test a difference in survival of 2% between the 2 groups at 5 years. 300 deaths were needed to trigger the overall survival analysis

38. Statistical analysis • How are you going to analyze your data once you get it? • Difference in proportions (what stats test?) • Difference in survival • Univariate (Kaplan Meier) • Multivariate adjusting for potential confounders (Cox proportional hazards)

39. Some issues in reporting results • What patients did you actually get? • Describe baseline characteristics (give p value?) • Effectiveness vs efficacy • NSABP B-32: “Analyses were done according to the random assignment of patients…” Effectiveness • drug trials • Effectiveness: Analysis performed according to assignment group • Efficacy: Analysis performed according to the intervention the patient actually received • Subgroup analyses • Numbers may be too small to detect real difference • Spurious significant p values • Rationale, limit number of analyses, a prior questions

40. Additional issues • Non inferiority vs superiority • NSABP B-32: non inferiority trial • “trial was designed to establish whether SLN resection achieves the same therapeutic outcomes as ALND” • “to test a difference in survival of 2% between the 2 groups” • Can never test for difference of 0 because sample size would be infinity; set mininum difference • Requires larger sample size than superiority trial • Interim analyses • Pre-set p values for each interim analysis; overall p value for the trial must be 0.05 • Early stopping • To avoid toxicity or delayed benefit if one arm appears significantly superior to the other • Early results may reverse by the end of the trial

41. Clinical vs statistical significance • Weaver et al, NEJM 2011;364:412-21 • Sentinel nodes examined at deeper levels with HE and IHC • Significant difference in overall survival (p=0.03) between patients with occult metastases and those without metastases • Adjusted HR 1.40 (95%CI 1.05-1.86) • 5 year OS • occult metastases 94.6% • no metastases 95.8% • “…differences…statistically significant, but relatively small…” • “Our findings argue against analysis of additional tissue levels or routine immunohistochemistry analysis for sentinel lymph node evaluation.”

42. Generalizability / external validity • Are patients similar to the study participants? • Is the exposure similar?

43. Observational studies case control • Useful when event rare, prolonged delay between exposure and outcome • Identify by outcome • Patients with lung cancer vs patients without lung cancer • Control group: people in community?, hospitalized patients for orthopedic surgery?, patients with CAD? • Look for smoking exposure in both groups • Control for important confounders / matching • Analyze: is there is increased proportion of smokers among patients with lung cancer?

44. Hospitalized patients with lung cancer Patients hospitalized for orthopedic surgery Exposure: current or past smoker? Exposure: current or past smoker? *Adjust for important variable such as age, etc.

45. Observational studies - Cohort • Start with exposure • DLBL and BCLU patients • Follow to outcome • Death due to any cause • Data can be gathered prospectively • Can control what data will be gathered • e.g. do c-myc testing in all patients as biopsies are received • Data can be gathered retrospectively • Data may be incomplete • E.g. c-myc testing done on some patients and report available; myc testing on others can still be done, but may not have adequate tissue in some

46. Cohort Population Exposure (histology) DLBC BCLU Outcome : Overall survival Outcome : Overall survival

47. Cohort: Itemized checklists • Strobe statement • Similar to Consort, but no randomization • “variables: clearly define all … exposures, predictors, potential confounders and effect modifiers” • REMARK (reporting recommendations for tumor marker prognostic studies) • Similar to Strobe, emphasis on describing the prognostic marker, defining other predictors of outcome

48. Dealing with confounders • If a potential confounder / predictor is known and can be measured, the analysis requires either a statistical adjustment for the confounder / predictor or subgroup stratification

49. Clinical characteristics

50. Overall Survival for DLBL vs BCLU HR2.5 (95% CI 1.2-5.2) Adjusted for IPI and treatment