An Introduction to Systematic Reviews VS Literature Reviews, PICO and The Cochrane Collaboration

1. An Introduction to Systematic Reviews VS Literature Reviews, PICO and The Cochrane Collaboration Mary Ellen Schaafsma Cochrane Collaboration & Nancy Roberts SHRTN Library Service – Eastern Ontario April 2008

2. Outline • Rationale for systematic reviews • Similarities & differences between other types of reviews and Cochrane Systematic Reviews • PICO and the Building Block approach to doing research • The Cochrane Collaboration and The Cochrane Library • Systematic reviews in other industries

3. Information Overload • Over 20,000 health journals published/year • Large quantities of information • To keep current GPs must read 19 new scientific articles per day (19 x 1 hr critical appraisal = 19 hrs/day) (Davidoff F et al BMJ 1995;310:1085) • Decision makers need to integrate the critical pieces of available evidence

4. Problems of information management • Published research is of variable quality and relevance • Health care (and other) professionals are often poorly trained in critical appraisal skills • Individual studies rarely by themselves provide sufficient evidence for policy or practice changes

5. Knowledge Synthesis: a solution • Synthesis takes large quantities of information and integrates them into one report • There are a variety of methods of synthesis, some more reliable than others • Systematic Reviews are efficient and scientific (adding rigour and thus reliability) • Less bias than a single study

6. Knowledge synthesis: a solution • Summarises evidence from a broad range of research methods addressing different questions. • Effectiveness: Does changing X change Y? • Relationships: Is X associated with Y? • Mechanisms How/why does changing X change Y? • Meanings: How are X or Y viewed or experienced?

7. Realist, Scoping and Literature Reviews

8. Narrative, Systematic Review, Cochrane Review

9. What is a systematic review? A review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyse data from the studies that are included in the review. Cochrane Collaboration (2005) Glossary of Terms in The Cochrane Collaboration

10. Systematic reviews - method A standardized approach: • Objectives • Selection criteria for included studies • Search strategy • Data collection and analysis • Report main results • Authors' conclusions • Plain language summary

11. Systematic reviews - advantages • Scientific approach • Less bias than single studies • Generic method to answer questions • Provides the ‘big picture’ synthesis of existing research • Identifies gaps in research and knowledge

12. Steps of a Systematic Review Grey & published, in all languages Systematic manual searches of key journals Computerized Databases Review of reference lists Consultation with experts Identify Studies Relevant Review for Relevance Not Relevant Evaluate Methodological Quality Reject Extract Data Analyze Data 12 Draw Conclusions

13. Meta-analysis – pooling results The use of statistical techniques in a systematic review to integrate the results of included studies. Cochrane Collaboration (2005) Glossary of Terms

14. The strength of a Cochrane Review • Rigour of methodology • Broad scope of literature included • Updated and maintained • Inclusiveness of perspectives • Plain language summary • Independence from commercial interests

15. How are systematic reviews used? • Evidence base for: • Health technology assessments • Clinical practice guidelines • Policy decisions about health care delivery and health systems • Focus research $: • Identify current knowledge gaps • Prioritise research funding • To add to the empirical evidence of systematic review methodology

16. What is evidence-based research? • The process of systematically locating, evaluating, interpreting and applying research outcomes to assist decision-making • Initially applied to clinical medicine, the concept has spread to include other areas of health care including management and caregiving activities. • Evidence informed decision-making • Use the evidence in conjunction with the uniqueness of the situation or client care need

17. What is the process of doing Evidence-based Research? • Formulate a sensible, focused question. • Search the literature for evidence related to the question. • Rate the quality of the available studies. • Apply the evidence to a particular situation • Assess outcomes of decisions. • Adapted from: http://hlwiki.slais.ubc.ca/index.php?title=Evidence-based_health_care#Five_.285.29_steps_of_EBM

18. What is the Building Block approach to basic research? Simply, it is a methodical approach to organizing your thoughts and to assist you in your search of the literature. 1 - Formulate The Question using PICO 2 - Identify Concepts 3 - Think Of Words/Phrases To Describe Your Concepts 4 - Connect Words And Concepts Using Boolean Logic (AND or OR) 5 - Review results from the databases searched and refine concepts, phases or question as needed

19. A well-formulated question is based upon a PICO statement • Patients or population (include conditions here) • Interventions (can be diagnostic or therapeutic, or “watchful waiting”) • Comparisons • Outcome Any evidence-based research should have at least 3 of the 4 parts of a PICO statement

20. 2 Questions, only one is well-formulated X What do you have on devices to prevent falls? √ What is the efficacy of the use of bed or chair alarms in preventing falls in the elderly?

21. What is the efficacy of the use of bed or chair alarms in preventing falls? Patient: frail elderly Intervention: bed or chair alarms Comparison: Use of alarms or not (efficacy of) Outcome: fall prevention Any research question should have at 3 components of a PICO statement

22. Words/Phrases For Your Concepts Concept 1: bed or chair alarms (specifically) Concept 2: efficacy or effectiveness, cost- benefit analysis Concept 3: accidental falls or fall prevention, patient safety

25. Example from audience • The program context is a community-based diabetes education program for under-serviced, vulnerable populations (e.., ethno-cultural groups; people living with mental illness; homeless people; etc). The review is in scientific and grey literature, searching these two terms (self-efficacy & community capacity), resulting in a list of available measurement tools or processes

26. What is the Question? Think about PICO? • What effective methods are used to evaluate how effective a community-based diabetes education program is in changing patient self-efficacy or self management?

27. PICO • P – Diabetes Patients in the community • I – education program • C- effectiveness of models • O –self- efficacy or self management

28. Apply to the databases • Look at Cochrane for the effectiveness, CINAHL and Medline • Sample search strategy • P – 1. diabet* • 2. Patient OR client • I – 1.Education* OR Instruct* AND Program • 2. community • C –1. Model* OR Program* • 2.Effectiveness OR Efficacy OR Evaluation OR Measur* • O – Self-efficacy OR self management OR self-management • Link all of the concepts together with AND

29. But • Not all literature reviews neatly fit the PICO format but you can still use the Building block approach to doing the research. • Break the topic into a number of questions • Identify the key concepts and related keywords • Keep each concept a search line linking like terms with OR (Expands search results) • Link all concepts together with AND (narrows search results) • Keep it simple • Refine and revise as you look at your results • Use limits to narrow search criteria if need be

30. Introduction to The Cochrane Collaboration

31. The Cochrane Collaboration The Cochrane Collaboration is a unique worldwide organization that aims to help people make well informed decisions about health care by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of health care interventions.

32. Structure of The Cochrane Collaboration Review Groups 51 Centres 12 Steering Group Consumer Networks Fields 16 Methods Groups 13

33. www.thecochranelibrary.com

34. What is The Cochrane Library? • Main output of Cochrane Collaboration • Contains Cochrane reviews & other databases www.thecochranelibrary.com

35. Cochrane Library; Issue 2, 2008

36. The Cochrane Library: Access • Free to half the world’s population! • In Canada – lobby for national licence (http://nlccl.epetitions.net/) • Public access in SK, NS and NB • Many University medical libraries • CMA, CDA, CAOT, CNA, CPhA

37. Other organizations and industries adopting the Systematic Review methodology Campbell Collaboration http://www.campbellcollaboration.org/ Canadian Language and Literacy Network http://www.cllrnet.ca/app/webroot/fusebox/index.php?fa=Programs.systematic SCIE – Social Care Institute for Excellence www.scie.org.uk Lancaster University Management School www.lums.lancs.ac.uk Social Science Research Network http://www.ssrn.com/ EPPI Centre http://eppi.ioe.ac.uk/cms/

38. Helpful References • Pawson, R., Greenhalgh, T., Harvey, G. & Walshe, K. 2005. Realist review--a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. Jul, 10 Suppl 1, 21-34. • Arskey, H. and O’Malley, L. 2005. Scoping Studies: towards a methodological framework, International Journal of Social Research Methodology , 8, 1, 19-32. • Popay, J. Baldwin, S., Arai, L., Britten, N., Petticrew, M., Rogers, M. & Sowden, A. 2003. Narrative Synthesis in Systematic Reviews. ESRC Methods Briefing 22. www.ccsr.ac.uk/methods/projects/posters/documents/Popay.pdf

39. Mary Ellen Schaafsma, M.H.A. Executive Director, Canadian Cochrane Network and Centre 1 Stewart Street, Rm 227 Ottawa, ON K1N 6N5 Phone: (613) 562-5800 ext 5224 Fax: (613) 562-5659 mschaafs@uottawa.cawww.ccnc.cochrane.org OR Nancy Roberts Information Specialist, SHRTN Library Service – Eastern Ontario c/o SCO Health Service Library 613-562-4262 ext. 1595, 1-866-393-4877 seniorslibrary@scohs.on.ca or visit SHRTN.ON.CA Thank-you – For more information contact: