1 / 34

Statistics for the Interventionist

Statistics for the Interventionist. Gregory J. Dehmer, MD Professor of Medicine, Texas A&M College of Medicine Director, Cardiology Division Scott & White Clinic. Statistics means never having to say your certain. Gregory J. Dehmer, MD, FSCAI.

quinlan-gregory
Download Presentation

Statistics for the Interventionist

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Statistics for the Interventionist Gregory J. Dehmer, MD Professor of Medicine, Texas A&M College of Medicine Director, Cardiology Division Scott & White Clinic Statistics means never having to say your certain

  2. Gregory J. Dehmer, MD, FSCAI I have no relevant financial disclosures to make.

  3. Expectations - Hypothesis p=1.0 • You will completely understand statistics at the end of this brief talk • This is inherently boring materialand I can’t fix that • Provide some comments about studies and statistics that I hope are helpful p<0.000000001

  4. Some Things Are Just Really Bad Ideas

  5. Another Questionable Combination + The busy interventional cardiologist Statistics-in-a- Box “SYSTAT has every statistical procedure you need” Hazardous equipment – Don’t operate unless you know what you are doing

  6. You Will Be Reviewed • Many papers now undergo formal statistical review Statistical Consultants NEJM Circulation Circ Res JACC etc . . .

  7. Evolution of Evidence Primary Evidence Secondary Evidence Randomized controlled trial Observational studies Uncontrolled trials Descriptive studies Case reports Synthesized quantitative Data (meta-analyses) Systematic reviews Summary reviews Opinions of respected authorities

  8. Evolution of Evidence Primary Evidence • Issues related to RTCs • Exclusions • Missing data • Power calculations • Confidence intervals • Confusing endpoints • “Non-inferiority” Randomized controlled trial Observational studies Uncontrolled trials Descriptive studies Case reports

  9. RTCs Problem 1: Exclusions • Exclusion of cases is a major weakness (during analysis) • Most common reason is a desire to ensure that all patients are “adequately treated” • Awkward to retain a patient in the analysis who died during the 1st week of therapy or were unwilling to stick with the therapy Your Out ! !

  10. RTCs Problem 1: Exclusions Exclusions more likely in: - the “aggressive treatment” arm or the “high-risk” group Standard Therapy R More aggressive than standard therapy • Likely to have more non-adherent patients • Non-adherent patients are a higher risk group • Exclusion of high-risk patients improved the average of the remaining patients • If exclusions are permitted, the more aggressive arm appears artificially better

  11. RTCs – The Problem With Exclusions Ideally, data analysis should look forward from randomization Good Events Good Events R Bad Events Side-Effects Dropouts These are the data (warts and all) that the clinician needs to know to assess what will happen to their patient if this therapy is selected

  12. RTCs – The Problem With Exclusions Data analysis when cases of inadequate treatment are excluded it like looking backward through rose-colored glasses Good Events Good Events R Data Analysis Bad Events Side-Effects Dropouts Excluding bad events and focusing only on the good results of the remaining cases may look impressive, but is not of practical value to clinicians who need to make prospective therapy decisions for their patients.

  13. RTC Lesson 1 – Avoid Exclusions Check to see if the size of the analyzed groups are similar R Standard therapy New therapy n = 4932 n = 4931 Exclusions n = 4932 n = 4100 Beware of potential bias

  14. RTCs Problem 2: Missing Data Missing random data weakens the study, but is not a serious concern However when data are missing because of aspects of treatment or disease, major bias can arise. Patients missing outcomes observations are more likely those with poor outcomes Higher-risk ptsdon’t tolerate the therapy, drop outleaving the low-riskpts who naturallyhave higher EFs LVEF # of patients: 200 120 50

  15. RTC Problem 2 - Beware of Missing Data • Make every effort to have data values at all key time points • Can use “imputed values” • Carry previous measure forward • Inserting a conservative value • Averaging adjacent values • Computer models which use similar patients with complete information

  16. RTC Problem 2 - Beware of Missing Data • Sensitivity analysis: determines the impact of the missing data and the imputation method used. • If the results are qualitatively similar, one can deduce that the basic study conclusion does not depend on the type of imputation used (or the use of imputation). Treatment 2 Treatment 1 Replace missing values with an anti-conservativeimputation Replace missing values with a conservative imputation Analyze and flip-flopimputation strategy

  17. RTC Lesson 2 – Avoid Missing Data Rule of ThumbIf the proportion of cases excluded or with missing data in  the sizeof the treatment difference reported,the study is likely unreliable Consolidated Standards for Reporting Trials Lancet. 2001;357:1191–1194

  18. RTCs Problem 3: Power Calculations • Power calculation • Determine what is a clinically meaningful difference between the two groups. (10%)Would anyone care if the difference in restenosis was 2%? • Amount of variation in the measurement of the endpoint (standard deviation)

  19. RTCs Problem 3: Understanding Power • One-tailed (sided) test • Used when previous data, physical limitations or common sense tells you that the difference, if any, can only go in one direction • Example: Contrast effect on renal function • Two-tailed (sided) test • Used when the difference, if any, can go in either direction. • Example: Drug effect on serum K+

  20. RTCs Problem 3: Understanding Power • Power Calculation: • - Null hypothesis %RS (control) - %RS (treatment) = zero (0) If you reject the null hypothesis then you are saying there is a difference between the two • = threshold of significance typically ≤ 0.05 (5%) If you reject the null hypothesis when it is actually trueType I error • There is a ≤ 5% chance that there no difference, but your analysis concludes there is Probability of a Type I error = 

  21. RTCs Problem 3: Understanding Power • Power Calculation: • = threshold of significance typically ≤ 0.05 Saying there is a difference when there is none •  =the level you are willing to accept for the chance of missing an important difference when there really is one (20%) (Type II error) Accepting the null hypothesis when it is, in fact, falsePower = 1 -  1 – 0.20 = 0.80 or 80%

  22. RTC Lesson 3 – Know What Power Means

  23. RTCs Problem 4: Understanding CIs • Standard deviation • Relates to one data set • Fasting cholesterol of everyone in this room • Mean (average) • SD is an expression of how much spread there is around the mean value • Equation for SD SD mark the limits of scatter Approximately 68% are within 1 SD Approximately 95% are within 2 SD

  24. RTCs Problem 4: Understanding CIs • Confidence intervals • Relate to populations (consider this room a population) • Measure cholesterol in a sample of the population (n = 10) • How well does the sample mean represent the population mean? • 95% CI tells you that the mean of the population has a 95% chance (19 out of 20 times) of being within the range of the sample mean

  25. RTCs Problem 4: Understanding CIs

  26. RTCs Lesson 4: Know Your CIs • Confidence intervals • Each sample has a mean and SD • SEM = SD/ √n • The 95% CI is ± 1.96 x SEM • There is only a 5% chance that this range of values excludes the true population mean value Variable

  27. RTCs Lesson 5: OR & RR Are Not the Same The PURSUIT Trial The primary endpoint (composite of death or MI at 30 days) was comparedin patients receiving eptifibatide vs. placeboEptifibatide group: 672 out of 4722 reached the primary endpoint Placebo group: 745 out of 4739 reached the primary endpoint Odds Ratio Odds in E: 672 / 4050 = 0.166 Odds in P: 745 / 3994 = 0.187 Odds ratio: 0.166 / 0.187 = 0.899 Risk Ratio Odds in E: 672 / 4722 = 0.142 Odds in P: 745 / 4739 = 0.157 Odds ratio: 0.142 / 0.157 = 0.905 There is a separate, but similar mechanism for calculating CI for ORs and RRs The PURSUIT Investigators: NEJM 1998;339:436-443

  28. RTCs Problem 5: Ratio Confusion Relationship between ORs and RRs for studies assessing harm Each line on the graph relates to a different baseline prevalence, or event rate in the control group When the prevalence of the event is low, say 1%, the RR is a good approximation of the OR For example, when the OR is 10, the RR is 9, an error of 10% We can use this graph to get a grasp of how misleading it could be to interpret ORs as if they were RRs. Relative Risk Odds Ratio

  29. RTCs Problem 5: Ratio Confusion Relationship between OR and RR for studies which are assessing benefit Each line on the graph relates to a different baseline prevalence, or event rate in the control group When event rates are very low the approximation is close, but breaks down as event rates increase For example, if the event rate is 50% and there is a 20% reduction in the odds, the relative risk adjustment will be little over 10% Relative Risk Odds Ratio

  30. RTCs Problem 6: Confusing Endpoint • Superiority trial: Designed to test for a statistically significant and clinically meaningful improvement (or harm) from the use of the experimental treatment over the usual care. Not different Superior Superior Superior Experimental Treatment Better Control Treatment Better

  31. RTCs Problem 6: Confusing Endpoint • Equivalence trial: Evaluates whether the difference in outcome for the experimental treatment compared with standard care falls within the boundary of a clinically-defined minimally important difference (MID) MID Clinically and statistically equivalent Neither clinically nor statistically equivalent Had these come from a superiority trial they would be clinically equivalent, but statistically inferior (#2) or superior (#1) Statistically equivalent Statistically equivalent Experimental Treatment Better Control Treatment Better

  32. RTCs Problem 6: Confusing Endpoint • Noninferiority trial: Results are evaluated assuming that the experimental treatment is not worse than the standard treatment by a clinically-meaningful amount. MID Not inferior CI too wide for any conclusions CI does not crossthe MID CI crosses MIDindicating inferiority of the experimental Rx Not inferior Experimental Treatment Better Control Treatment Better

  33. Resources • http://www.jr2.ox.ac.uk/bandolier/ • http://www.statsoft.com/textbook/stathome.html • http://www.bettycjung.net/Statsites.htm • http://www.tufts.edu/~gdallal/bmj.htm • Link to Br Med J series of papers on statistics • 2006-2007 Circulation series “Statistical Primer for Cardiovascular Research” • Motulsky H. Intutitive Statistics. Oxford University Press 1995 A statistician is a person who comes to the rescue of figures that cannot lie for themselves

  34. Remember Statistics are like a bikini. What they reveal is suggestive, but what they conceal is vital

More Related