Marguerite Koster, MA, MFT* Practice Leader Technology Assessment and Guidelines Unit

1. Marguerite Koster, MA, MFT* Practice Leader Technology Assessment and Guidelines Unit Kaiser Permanente Southern California *In collaboration with Wiley Chan, MD (KP Northwest) and Jeffrey Harris, MD (KP Northern California) Using Externally Developed Clinical Practice Guidelines: Barriers and Opportunities Institute of MedicineCommittee on Standards for Developing Trustworthy Clinical Practice GuidelinesWashington, DCJanuary 11, 2010

2. A Prepaid Integrated Health Care Delivery System Permanente Medical Group Kaiser Permanente Kaiser Foundation Hospitals Kaiser Foundation Health Plan About Kaiser Permanente • Social Mission • Quality Driven • Shared Accountability for Program Success • Integration along Multiple Dimensions • Prevention & Health Maintenance Focus

3. About Kaiser Permanente HI Description: People: Facilities: Revenue: Nation’s largest nonprofit health plan (founded 1945) Integrated health care delivery system 8.7 million members 14,000+ physicians 160,000 employees 8 regions in 9 states and D.C. 32 hospitals and med centers 420+ medical offices $38 billion annually WA OR NCal OH CO MD DC VA SCal GA

4. KP National Guideline Program (NGP) NGP identifies, develops and maintains a core set of 19 evidence-based clinical practice guidelines 9 related to preventive care (immunizations, screening, counseling) 10 related to chronic conditions management National Guideline Directors (NGD) oversees all NGP efforts at least one representative from each region sponsored by KP’s Care Management Institute Guideline Development Resources funded internally goal is to use high-quality external systematic evidence reviews and evidence-based guidelines when available

5. KP Guideline Development Process Guideline Development Teams: Participants from all KP Regions Physicians: Clinical lead, methodologist, MDs from multiple specialties (represent all stakeholder groups potentially impacted by guideline) Nonphysicians: Analytical staff (MPH), pharmacists, other clinicians KP’s internally developed “Common Methodology” establishes the criteria for guideline development Clearly defined clinical questions (Patient, Intervention, Comparison, Outcomes) Systematic evidence search (use existing high-quality SRs when available) Critical appraisal of evidence in narrative and evidence table formats Evidence grading and labeling of recommendations Explicit, compelling rationale for consensus recommendations Guideline Quality Committee Reviews all documentation for consistency with Common Methodology Medical editor assures readability and logic, cross check on grading

6. Barriers to Use of External Guidelines Wide variation in guideline development processes and products; many definitions of “evidence based” Lack of transparency of CPG processes and products Lack of rigorous standards for evidence-based guideline methodology Lack of standardized format and content Little explanation of processes for consensus-based recommendations when evidence is absent/poor Few actionable, implementable recommendations that can be embedded in electronic systems for use at the point of care Absence of an oversight group to monitor guideline quality Few collaborative efforts between between guideline developers and users

7. Variation in guideline development processes and products There is significant variation in the development processes and products of guideline developers Areas of significant variation include: Use of and framing of clinical questions Thoroughness and documentation of evidence searches Abstraction and documentation of data from studies Critical appraisal and grading of individual studies and bodies of evidence Systems for grading quality/strength of recommendations General format and content (e.g., “evidence-based” guidelines with no documentation) Panel composition, training, and potential conflicts  Qualifications of staff and panel members

8. Guideline Transparency Guideline users/consumers need to be able to see all steps of the systematic review and guideline recommendation processes, including: Clinical question(s), evidence search terms and results Full documentation, analysis, critical appraisal and grading of the evidence Grading of the quality and strength of guideline recommendations Panel membership, training, affiliations and conflicts of interest Documentation of panel discussions/decisions, disagreements, minority views, value judgments and perspectives that shaped the guideline recommendations

9. Rigorous, Evidence-Based Guideline Development Standards High-quality guidelines are built on high-quality systematic evidence reviews Need to encourage adoption of evidence-based methodology and quality standards by guideline producers and users/consumers Developing and documenting clinical questions Conducting a comprehensive, systematic evidence search, including documentation of search terms and results Data abstraction and study documentation/display (evidence tables) Critical appraisal of individual studies and body of evidence Need for standardized grading of recommendations Endorse grading systems that incorporate assessment of quality of evidence and strength of recommendation (e.g., GRADE, AHRQ) Careful balance of benefits and harms Explicit declaration of perspectives, values and subjective judgments Consideration of efficient, effective resource use

10. Standardized Guideline Format/Content Lead with well-defined clinical questions (e.g., define patient, intervention, comparison, outcomes) Provide documentation of systematic evidence review Search strategy and results, data abstraction/display (evidence tables), summary of critical appraisal and evidence grading Include as a separate companion document to guideline Rationale Statements Well documented summary of logic from clinical question to final recommendations Statement of values underlying recommendations Logical progression from diagnosis through treatment Separate basic science and didactic material from evidence-based clinical guideline information

11. Consensus-Based Recommendations Require a compelling rationale to justify development of consensus recommendations in the absence of moderate or good evidence If there is a compelling rationale: Use a formal, explicit decision-making process Agree on generally accepted rules to guide the process (e.g., if no evidence of benefit, but some evidence of harm, don't do it) Involve a trained facilitator/methodologist and expert panel with multidisciplinary representation Consider issues related to resource stewardship and patient preferences/shared decision making Document the process and decisions, including minority views

12. Guideline Development Panels Composition:Representation from diverse stakeholders, at least one well-trained evidence-based methodologist, evidence analysts (physician or nonphysician) Training: Methodology training for panel members to increase quality/consistency of recommendations Basic evidence-based medicine principles, study examples, evidence grading, recommendation development exercises, etc. Process Rules:Explicit “rules of order” to resolve evidence issues, small group domination, hold-outs, filibusters, etc. Disclosure requirements: All panel members and reviewers Affiliations and conflicts of interest Financial or other support for guideline development

13. Actionable, Implementable Guideline Recommendations Write unambiguous, fully specified, action-oriented recommendations that can be accessed efficiently at the point of care Develop fewer recommendations; focus on key levers in the process of care Embed guidance into an efficient workflow using electronic health information system tools, such as best practice alerts, panel management systems and “e-Guidelines”

14. Foster Guideline Collaboration Foster collaboration between systematic reviews, guideline developers and guideline users/consumers in the U.S. Prioritize and coordinate topics among guideline developers and users/consumers Avoid duplication of guidelines on same topics Ensure that the most important clinical topics affecting health care practitioners are addressed Learn from/build on existing efforts Potential to target funding and analytical resources for guideline development to the most important clinical topics