Critical Appraisal of the Biomedical Literature

1. Welcome Critical Appraisal of the Biomedical Literature Peggy Edwards, AMLS TTUHSC - Preston Smith Library Lubbock, Texas 79430 June 2012

2. Objectives During this learning module, participants will: 1) know the definition and steps in critical appraisal 2) become familiar with the hierarchy of study design 3) identify the appropriate study design for therapy, diagnosis, etiology/harm, and prognosis 4) know how to categorize the levels of evidence 5) learn the definition of “N-of-1 Randomized Controlled Trials” 6) critically appraise an article using a JAMAevidenceworksheet 7) search a practice topic in PubMed’s Clinical Queries

3. Steps in the Evidence-Based Process are: This module discusses critical appraisal (Citrome, & Ketter, 2009)

4. What is Critical Appraisal? “The process of assessing and interpreting evidence by systematically considering its validity, results, and relevance.” The Cochrane Collaboration Glossary http://www.cochrane.org/glossary/5#letterc

5. “Critical appraisal is the first step in transferring research knowledge into practice.” Das, K., Malick, S., & Khan, K. (2008). Tips for teaching evidence-based medicine in a clinical setting: lessons from adult learning theory. part one. Journal of the Royal Society of Medicine, 101(10), 493-500. doi: 10.1258/jrsm.2008.080712 web address: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2586873/?tool=pubmed

6. Purpose of Critical Appraisal “Randomized controlled trials and systematic reviews are the highest levels of evidence but they are not automatically of good quality and should always be appraised critically.” Das, K., Malick, S., & Khan, K. (2008). Tips for teaching evidence-based medicine in a clinical setting: lessons from adult learning theory. part one. Journal of the Royal Society of Medicine, 101(10), 493-500. doi: 10.1258/jrsm.2008.080712 web address: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2586873/?tool=pubmed A Consultation of Surgeons, old negative no. 62-116 [engraving]. [Bethesda, MD., United States of America]: Images from the History of Medicine, U.S. National Library of Medicine, National Institutes of Health; accessed June 12, 2012. The National Library of Medicine believes this item to be in the public domain. print: 11 x 16 cm.

7. Clinical Questions and Study Design “There are 5 fundamental types of clinical questions: 1) therapy 2) harm 3) differential diagnosis 4) diagnosis 5) prognosis” (Guyatt, 2008) It is important “to correctly identify the category of study, because, to answer your question, you must find an appropriately designed study.” (Guyatt, 2008)

8. Study Category Suggested Best Method of Investigation Therapy RCT>cohort>case control>case series Diagnosis prospective, blind comparison to a gold standard Etiology/Harm RCT>cohort>case control>case series Prognosis cohort>case control>case series Prevention RCT>cohort>case control>case series Clinical Exam prospective, blind comparison to a gold standard Economic Analysis Cost Medical Library Association. MLANET, Education, Web-based Learning. Hp. Nov, 2001. Web-based Courses: EBM and the Medical Librarian. Available: http://www.mlanet.org/education/web/web_courses.html10 Apr. 2005.

9. Hierarchy of Study Design (TTUHSC Preston Smith Library, 11/21/2008)

10. Evidence–Based Treatment “Clinicians should use the results of randomized controlled trials (RCTs) of groups of patients to guide their clinical practice. However, clinicians cannot always rely on the results of RCTs … To determine the best care for an individual patient, clinicians can conduct n–of–1 randomized controlled trials in individual patients.” (Guyatt, 2008)

11. Hierarchy of Strength of Evidence for Prevention & Treatment Decisions  N–of–1 randomized trial  Systematic reviews of randomized trials  Single randomized trial Systematic review of observational studies addressing patient–important outcomes Single observational study addressing patient–important outcomes Physiologic studies (studies of blood pressure, cardiac output, exercise capacity, bone density, and so forth) Unsystematic clinical observations • (Guyatt, 2008)

12. N–of–1 Randomized Controlled Trials Experiment designed to • determine effect of an intervention/exposure on a single study participant In a one N–of–1 design • the patient undergoes pairs of treatment periods • 1 period involves the use of the experimental treatment • 1 period involves the use of an alternate treatment/placebo • if possible, patient and clinician are blinded • outcomes are monitored Treatment periods are replicated until clinician and patient are convinced that • treatments are definitely different • ordefinitely not different • (Guyatt, 2008)

13. Assignment Go to JAMAevidence • create a profile (there is no fee) • Read Chapter 6 Therapy (Randomized Trials) Locate the Therapy Worksheet in JAMAevidence • Evaluate the article using the worksheet

14. www.ttuhsc.edu/libraries Hover mouse on Databases Click Evidence Based Databases

15. Click JAMAevidence

16. Click My JAMAevidence

17. Create a Profile Account

18. Click Therapy

19. Read Chapter 6 Therapy (Randomized Trials)

20. Read this Article Sweeting, M. J., Thompson, S. G., Brown, L. C., Greenhalgh, R. M., & Powell, J. T. (2010). Use of angiotension converting enzyme inhibitors is associated with increased growth rate of abdominal aortic aneurysms. Journal of vascular surgery, 52(1), 1-4. doi: 10.1016/j.jvs.2010.02.264 PMID:20494541 To retrieve this article, use the instructions on the following screens:

21. www.ttuhsc.edu/libraries Mouse over Databases Click PubMed

22. Enter the PMID: 20494541

23. Click the Elsevier icon

24. The full-text article will open.

25. After reading the article, critically appraise it using the therapy worksheet from JAMAevidence. Use the following instructions:

26. From the page at the end of Chapter 6: Click Critical Appraisal Worksheet: Therapy

27. From the Home Page: Click My JAMAevidence

28. Click Therapy Worksheet

29. Critical Appraisal Worksheet - Therapy

30. Additional Information http://ktclearinghouse.ca/cebm/practise/ca

31. Applying the Results “Each (patient management) decision involves a consideration of the relevant evidence and a weighing of the likely benefits and downsides in light of the patient’s values and preferences. When considering choices, clinicians may benefit from structured enumeration of the options and outcomes, systematic reviews of the evidence regarding the relationship between options and outcomes, and recommendations regarding the best choices.”(Guyatt, 2008)

32. Quality and Levels of Evidence * Medical evidence or recommendations can vary in quality * Sources of evidence range from: 1. small laboratory studies to 2. well-designed large clinical studies with minimized bias * Is a recommendation is strong or weak? * Can you be confident in the recommendation? *Grading by strength of recommendations is a systematic approach which can minimize bias and aid interpretation * Quality of evidence can be categorized as high, moderate, low, or very low (The GRADE Working Group, 2005)

33. GRADE GradingofRecommendations,Assessment, Development, and Evaluation– Code Quality of Evidence Definition • Further research is very unlikely to change our confidence in the estimate of effect. • Several high–quality studies with consistent results • In special cases: one large, high–quality multi– • center trial A High • Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate. • One high–quality study • Several studies with some limitations B Moderate • Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. • One or more studies with severe limitations C Low • Any estimate of effect is very uncertain. • Expert opinion • No direct research evidence • One or more studies with severe limitations D Very Low (Essential Evidence Plus EBM Guidelines Editorial Team, 2010)

34. Strength of Recommendation "Recommendations to administer, or not administer, an intervention, should be based on the tradeoffs between benefits on the one hand, and risks, burden and, potentially, costs on the other. If benefits outweigh risks and burden, experts will recommend that clinicians offer a treatment to typical patients. The uncertainty associated with the tradeoff between the benefits and risks and burdens will determine the strength of recommendation." (The GRADE Working Group, 2005) StrengthOfRecommendationTaxonomy(SORT) Code Definition Consistent, good–quality patient–oriented evidence * A Inconsistent or limited–quality patient–oriented evidence * B *Patient–oriented evidence measures outcomes that matter to patients: morbidity, mortality, symptom improvement, cost reduction, and quality of life. Disease– oriented evidence measures: immediate, physiologic, or surrogate end points that may or may not reflect improvements in patient outcomes (e.g. blood pressure, blood chemistry, physiologic function, pathologic findings). C Consensus, disease–oriented evidence *: usual practice, expert opinion, or case series for studies of diagnosis, treatment, prevention, or screening • (Essential Evidence Plus EBM Guidelines Editorial Team, 2010)

35. Summary Topics/exercises covered in this section included: 1) the definition and steps of critical appraisal 2) the hierarchy of study design 3) the appropriate study design for therapy, diagnosis, etiology/harm, and prognosis 4) the levels of evidence categories 5) the definition of “N-of-1 Randomized Controlled Trials” 6) critical appraisal of an article using a JAMAevidenceworksheet

36. Assessment 1) What is the definition of critical appraisal? What is the purpose? 2) Name three high quality study designs. 3) What are the four categories of levels of evidence in the GRADE framework?

37. EBP Literature from PubMed’s Clinical Queries

38. PubMed Database • Biomedical journal literature • International scope • Published by: • The National Library of Medicine

39. Accessing PubMed • Freely available over the Internet @ www.pubmed.gov • Also through library’s home page

40. PubMed’s Clinical Queries tool that searches for evidence-based literature

41. Clinical Queries Searchable by: Diagnosis Therapy Prognosis (prediction of probable outcome) Etiology (cause)

42. PubMed’s Clinical Queries uses pre-formulated search strategies with a keyword search

43. www.ttuhsc.edu/libraries Mouse over Databases Click PubMed

44. TTUHSC Libraries - Proxy Server When Searching from Off Campus: Enter the Eraider username and password assigned to you by Information Services. A Problems with eRaider call: 806-743-2870 or 806-743-2875 a

45. Click Clinical Queries

46. Enter asthma Search Click

47. Results for Therapy and Broad Scope Change toNarrow Scope

48. Click See all (7542)

49. Click Limits

50. Click on the article title you wish to read