Preparing the Future Primary Care Workforce Together

1. Preparing the Future Primary Care Workforce Together Primary Care Faculty Development Initiative (PCFDI) CBME in the Ambulatory Setting Nov

2. Outline • CBME background • Key concepts and definitions • Frameworks and outcomes • Where we are/where we need to be • The role of milestones and entrustment in the assessment and evaluation of competence

3. Competency versus Competent Adapted from: The International CBME Collaborators, 2009 Competency: an observable ability of a health professional, integrating multiple components such as knowledge, skills, values and attitudes. Competent: demonstrating the required abilities in all domains in a certain context at a defined stage of medical education or practice.

4. What does competency-based medical education means to you?

5. Competency-Based Medical Education • is an outcomes-based approach to the design, implementation, assessment and evaluation of a medical education program using an organizing framework of competencies • Bottom line: • CBME = Outcomes-based Medical Education (OBME) The International CMBE Collaborators 2009

6. The Framework: ACGME Competencies • Medical knowledge • Patient care and procedural skills • Interpersonal and communication skills • Practice-based learning and improvement • Systems-based practice • Professionalism

7. Outcome?

8. The Outcome Frenk J, et al. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010

9. What is the outcome? A competent (at a minimum) practitioner aligned with: IOM Six Aims for Quality CMS Triple Aim National Priorities Partnership

10. Individual Physician Readiness: The Gaps • Office-based Practice Competencies • Inter-Professional team skills • Clinical IT Meaningful Use skills • Population management skills • Reflective practice and CQI skills • Care Coordination • Continuity of Care • Leadership and management skills • Systems thinking • Procedural Skills Crosson Health Affairs 2011

11. Is CBME/OBME Just a “Fad”? • Pet rocks • Leisure suits • Streaking • Disco music • Yugos • Pokemon • Tickle me Elmo …probably not…

12. Is CBME/OBME a Paradigm Shift? Thomas Kuhn (1962): “Normal science, the activity in which most scientists inevitably spend almost all of their time, is predicated on the assumption that the scientific community knows what the world is like. Much of the success of the enterprise derives from the community’s willingness to defend that assumption, if necessary at considerable cost” Thomas S. Kuhn. The Structure of Scientific Revolutions. University of Chicago Press. Chicago. 1962. Pg. 5.

13. Could the Same be True of UME and GME? • “Normal medical education, the activity in which most faculty inevitably spend almost all of their time, is predicated on the assumption that the medical educational community knows what the world is like. Much of the success of the enterprise derives from the community’s willingness to defend that assumption, if necessary at considerable cost” Thomas S. Kuhn. The Structure of Scientific Revolutions. University of Chicago Press. Chicago. 1962. Pg. 5.

14. Is CBME/OBME a Paradigm Shift? • Maybe…but perhaps that is not the main point: • CBME is yet another stage on what should be the ongoing evolution and improvement of medical education • The focus on outcomes is worthy of our attention

15. The Transition to Competency • Fixed length, variable outcome • Structure/Process • Knowledge acquisition • Single subjective measure • Norm referenced evaluation • Evaluation setting removed • Emphasis on summative Competency Based Education • Competency Based • Knowledge application • Multiple objective measures • Criterion referenced • Evaluation setting: DO • Emphasis on formative • Variable length, defined outcome • Caraccio et al 2002

16. Milestones The definition of expected outcomes or competencies

17. Asignificant point in development that identifies the discrete knowledge, skills, and attitudes expected of learners as they progress through training. Milestones should enable the trainee, program and the certification board to know an individuals trajectory of competency acquisition. Milestones

18. Dreyfus & DreyfusDevelopment Model PGY3 Expert/ Master PGY1 Proficient MS4 Competent MS3 Advanced Beginner Novice Time, Practice, Experience Dreyfus SE and Dreyfus HL. A 1980 Carraccio CL et al. Acad Med 2008;83:761-7

19. Reporting Milestones-IM

20. Sub-bullet “Curricular” Milestone

21. Provide the learner with a clear path of progression There are no surprises Allow for rich formative feedback. Learners know where they are and where they need to go Define specific behaviors that can focus assessment Milestones Benefits

22. Milestones are too reductionist Checklist = competence Checking off a milestones list does not equal competent practice in a highly complex health care environment Operationalize the milestones to develop and apply meaningful assessment and evaluation. Milestones Criticisms

23. Entrustment/Entrustable Professional Activities (EPAs) A framework for work-based assessment?

24. Entrustable Professional Activities EPAs represent the routine professional-life activities of physicians based on their specialty and subspecialty The concept of “entrustable” means: ‘‘a practitioner has demonstrated the necessary knowledge, skills and attitudes to be trusted to independently perform this activity.’’1 1Ten Cate O.Acad Med. 2007;82(6):542–547.

25. An Entrustable Professional Activity • Part of essential work for a qualified professional • Requires specific knowledge, skill, attitude • Acquired through training • Leads to recognized output • Observable and measureable, leading to a conclusion • Reflects the competencies expected… • EPA’s together constitute the core of the profession ten Cate et al. Acad Med 2007

26. “Entrustment in Medical Education” Focused assessments around what faculty and training programs already “entrust” trainees to do? Reflects the most important outcome of training: a trainee’s readiness to bear professional responsibility” Enables work-based assessment focusing on demonstrating competence in desired outcomes of training.

27. Learners: have Direct supervision Faculty member sees every patient Managers: have Indirect supervision Faculty member discretion to see patient Teachers: Oversight from faculty Resident discretion to allow patients to leave before precepting Baystate AmbulatoryLMT Model Adapted from Sudeep K. Aulakh & Michael J. Rosenblum. Presented at ICRE 2012, Ottawa.

28. Failure -Frequently does not use these skills Needs work -Inconsistently uses these skills Competent -Consistently uses Invite & Listen; Summarizes in a reporter fashion Proficient -Consistently uses all three skills, Summarizes interpreted information Expert -Consistently uses all three skills, Summarizes interpreted information in complex cases Ambulatory Milestone: Demonstrates patient-centered interviewing using the Invite, Listen, Summarize format Adapted from Sudeep K. Aulakh & Michael J. Rosenblum. Presented at ICRE 2012, Ottawa.

29. Competencies, Milestones and EPAs COMPETENCIES EPAs MILESTONES

30. The Synthesis – Analytic Tension • Physicians do not apply each competency independently in caring for patients • As a result, judging overall performance is a synthetic/integrative assessment activity However, • You will often have to pull things apart (analysis) to create shared mental models and to provide meaningful and actionable feedback to the resident

31. Lets watch a video. What has this resident been entrusted to do? If this were your institution, could you attest that the resident had the required competence to provide this care?

32. With Your Neighbor - Pick a goal of training that would meet a national priority for ambulatory-based care. Identify two to three entrustments in training that could serve as a focus for assessment in that competency? What assessment methods would you use? What process will you use to make an entrustment decision? What facilitators and barriers would you face?

33. The “System” Accreditation: ACGME/RRC Institution and Program Residents Program Aggregation • Assessments within Program: • Direct observations • Audit and performance data • Multi-source FB • Patient experience • Simulation • ITExam Judgment and Synthesis: Committee NAS Milestones Board Reporting No Aggregation Certification: Board Faculty, PDs and others Milestone and EPAs as Guiding Framework and Blueprint

34. Break