1 / 39

Critical Appraisal

Critical Appraisal. Evidence –Based Medicine. Formulate problem Track down best evidence Appraise the evidence critically Implement results in clinical practice Evaluate performance. All published studies in reputable journals:. Have the perfect design ?

oshin
Download Presentation

Critical Appraisal

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Critical Appraisal

  2. Evidence –Based Medicine Formulate problem Track down best evidence Appraise the evidence critically Implement results in clinical practice Evaluate performance

  3. All published studies in reputable journals: Have the perfect design ? Use the most appropriate statistical analysis? Present the results in the best way?

  4. Design Issues

  5. Methodological issues I • How were the subjects recruited? • Who was included/excluded ? • What justification is given for the sample size? • Was bias avoided/minimised?

  6. “A sample of patients were interviewed within the first week of their admission to hospital. 46 people were assessed, representing 15% of admissions over the study period”

  7. Early trials of anti-histamine agents for seasickness during World War 1 All soldiers crossing the Atlantic on Ship A had active drug All soldiers crossing the Atlantic on Ship B had placebo

  8. “The main contribution of statisticians in medical research is not to carry out statistical analyses but to inject a bit of logic into the situation”

  9. Methodological issues II • Was random allocation to treatment used? • Was potential degree of blindness used ? • What was the duration and completeness of follow-up?

  10. Example 1“Prevention of wound infection”

  11. Randomisation techniques In a randomised controlled trial: “Patients were allocated to either a control or intervention group using date of birth”

  12. Randomisation Date of birth ‘randomisation’ = Systematic allocation (not randomisation) Open to bias

  13. RCT ‘programming error’ in allocation of treatment

  14. Presentation of Results

  15. Descriptive Information • Summary statistics (proportions, means, medians, ranges, standard deviations) • Characteristics of non-respondents • Characteristics of withdrawals from clinical trials

  16. “…mean (sd) of social class = 3.2 (1.2)”

  17. Baseline characteristics

  18. Statistical Analysis

  19. “We have chosen to use parametric statistics to stay in line with the data analysis reported in the literature”

  20. 5-year survival rate after pancreatic resection Treatment A (n= 8) 25% Treatment B (n=19) 58% (p=0.35)

  21. Example 2“Patients with peripheral circulatory disorders”

  22. Report from Danish Newspaper Impressive improvement in crime statistics 1983-1984 Number of reported crimes reduced by almost 50%

  23. Report from Danish Newspaper In 1983, a man had made nuisance phone calls to the same woman (199 times) Each had been reported and counted separately The man was apprehended early in 1984

  24. Data Torturing

  25. “Opportunistic”[Fishing expeditions] Pore over data until a ‘significant’ association is found Devise a biologically plausible hypothesis to fit the association

  26. Study of parents’ occupational exposures as a risk factor for birth defects in their offspring • Seven major categories of occupation identified • No significant relationship (for mothers or fathers)

  27. Study of parents’ occupational exposures as a risk factor for birth defects in their offspring Categories split into: • 64 separate occupations for mother • 80 for fathers • 5 significant relationships found

  28. “Procrustean” Deciding on the hypothesis to be proved and matching the data to fit the hypothesis ie. selective reporting [Procrustes, a robber in Greek mythology, made all his victims fit the length of his bed by stretching or cutting off their legs]

  29. Exposure redefined to strengthen the association In a study of adverse effects of oral contraceptives on the outcome of pregnancy Exposure defined as OC used within 600 days before a delivery or miscarriage

  30. Exposure redefined to strengthen the association Why 600 days? Why not 365 days (ie. 1 year)? Or 2 years? Or 18 months? Or 500 days?........

  31. Conclusion justified?

  32. Example 3“Exposure to X-rays”

  33. Hawthorne Effect

  34. Hawthorne Effect • Study by Western Electric Company to assess effects of illumination on production at the Hawthorne plant in Chicago • Control group – worked under constant illumination • Experimental group – worked under varying illumination

  35. Hawthorne Effect • Production increased in experimental and control groups at same rate

  36. Hawthorne Effect • Production increased in experimental and control groups at same rate Increases in production were caused by the increased attention that workers received from management

  37. Example 4“Bran study”

  38. Regression to the Mean Sir Francis Galton’s publication: “Regression towards mediocrity in hereditary stature” Very tall parents tend to have children who are not quite as tall, while short parents tend to have taller children

  39. Conclusion:Never believe everything you read

More Related