Presentation Transcript

1. Site Visit 102: The DIO’s Role for Program Site Visits

   Ann M. Dohn, M.A. DIO Stanford
2. Disclaimer No Conflicts of Interest to Report
3. DIO Competencies Maintaining the Institution’s Residency Program’s ACGME accreditation Improving the Institution’s education program Developing and supporting residency program directors
4. DIO Vision “Your job is not a powerful one…your job is to empower others.” i.e. program directors Helen Rantz, 1984
5. Session Objectives To understand the role of the DIO in ACGME program site visits Using ACGME annual resident and faculty surveys Using internal reviews and annual program evaluation in preparation for site visits Using other potential measures of program quality
6. Session Objectives A time of change… Living in two worlds-starting 7/1/2013 “PIF-less Site visits / NAS Site visits with PIFs ACGME OLD NEW
7. Then and Now 1984 site visits vs. 2012 site visits
8. Start with the New and “unknown” Unknown is scary
9. New Model: What Do We Know? Longer Cycles Annual WebADS Updates ACGME Resident and Faculty Surveys End of the PIF
10. End of the PIF DIOs will have more recreational reading time!
11. “New” Model Effective 7/1/2013 “Early adaptors”= RRCs deciding to go “PIF-less” Emergency Medicine Internal Medicine Neurosurgery Orthopedic Surgery Pediatrics Radiology (Diagnostic) Urology and their subs
12. New Accreditation System (NAS) The NAS requires categorization of the ACGME’s Common Program Requirements to clearly identify: Core Requirements Detail Requirements Outcome Requirements
13. New Accreditation System (NAS) Core Requirements Statements that define structure, resource, or process elements essential to every graduate medical educational program. Duty Hours Sponsoring Institution’s responsibility for Program Accreditation Change in program director Qualifications of the program director
14. New Accreditation System (NAS) Detail Requirements: Statements that describe a specific structure, resource, or process for achieving compliance with a Core Requirement.
15. Examples of “Detail” Requirements Detail Requirements include particular educational approaches and learning experiences a given number of lectures or hours of bedside teaching experiences Ensuring compliance with grievance and due process procedures as set forth in the Institutional Requirements and implemented by the sponsoring institution
16. New Accreditation System (NAS) Outcome Requirements: Statements that specify expected measurable or observable attributes (knowledge, abilities, skills, or attitudes) of residents or fellows at key stages of their graduate medical education. Programs in substantial compliance with the Outcome Requirements may utilize alternative or innovative approaches to meet Core Requirements.
17. Example of “Outcome” Requirements Outcome requirements include: Milestones Residents must be able to competently perform all medical, diagnostic, and surgical procedures considered essential for the area of practice.
18. Common Program Requirements
19. Neurosurgery – new Accreditation Example…
20. New Model-Tracer Model-Inverted Model Increases value on resident input ACGME survey Resident interviews Focuses on PROGRAM rather than a description of the program The Tolstoy novel will no longer be written!!
21. New Model-PIF Questions Small number of questions from PIF now in annual WebADS update Remaining information assessed by the field staff
22. Tracer Method Allows for the selection of particular elements to be evaluated: Citations Complaints from residents/faculty Non-compliance reported on the ACGME resident survey Duty hour and learning environment standards
23. Tracer Method Focuses on: Program’s actual operations Implementation of policies and procedures
24. Tracer Method – Interactive – A Key Change Allows for discussion of important topics related to the new duty hour standards Provides opportunities for site visitors to educate program leaders and share innovative practices
25. Tracer Method Longitudinal information on: New standards Prior citations/corrective actions Resident survey identified areas of non-compliance Areas identified during preparation for the Site Visit
26. Tracer Method Will focus on relevant high-priority issues for the given program Planned for two-four areas/topics per program Done during the regular interview (no walkarounds)
27. Tracer Method Enhancing resident input Residents to directly respond to the site visitor with a single, confidential “consensus list” strengths areas for improvements starts with the trainees
28. PIF-Less Site Visit: The Stanford Experience August 2011 Three Programs Visit done in lieu of mid-cycle internal review Data not used for accreditation Focus was on program requirements Used comprehensive data system (no paper)
29. PIF-Less Site Visit: The Stanford Experience (2) Individuals interviewed = regular site visit ACGME provided comprehensive reports Extensive debriefing with the site visitors and program participants Very positive experience for PDs. PCs, and the DIO and GME staff
30. PIF-Less Site Visit: Stanford Findings PIF-Less (inverted) site visit report finding Comparable to standard internal review Note: our protocol always starts with the residents/fellows
31. PIF-Less Site Visit: ACGME Findings Findings: Residents feel more engaged PDs love it Early ID of Program Themes Improves continuity and integration of data Seamless process for follow-up Increased opportunity for reconciliation of discrepant issues Transformative Model - rather than additive
32. PIF-Less Site Visit: ACGME Findings (2) Findings Continued Comparable to visit with PIF for citation review Superior for review of program improvements and positive program elements Variability with other elements Dependent on data system used by the institution/program
33. Upcoming Changes in the Site Visit Broader implementation of the inverted Site Visit Team site visits (two site visitors) Programs Institutions
34. Yikes! ACGME is coming….
35. ACGME Use of Survey Data during the Site Visit Use resident data to validate/clarify program information Compare resident data to program director and faculty interviews Validate resident data to program documentation Compare duty hours responses to institutional data
36. DIO Use of Data for Program Site Visits ACGME resident and faculty surveys APRs Internal Reviews Other measures of program quality
37. Annual ACGME Surveys2012
38. Annual ACGME Surveys2009
39. ACGME Concerns with the Resident Survey ACGME concerns with the resident survey: “Honesty” in reporting ? Residents being coached ? Amount of time spent by residents completing the survey = 20 seconds per question Survey questions being sold to companies who want to ‘help’ programs prepare Survey fatigue
40. ACGME Annual Resident Survey Possible solutions to using the survey 2012- Aggregation of individual survey questions into groupings 2012-use of alternate forms of the survey Educating residents about the vital nature of the survey
41. Annual Program Reviews: Summary
42. Annual Program Reviews: Data Elements
43. Internal Reviews
44. Monitoring Internal Reviews
45. Other Possible Sources of Data GME Annual House Staff Survey Duty Hours Reports Evaluations of programs, faculty, and house staff Alumni surveys Board and in-training exam scores
46. GME Annual House Staff Survey Provides feedback on eight (8) key areas of Graduate Medical Education: Overall Experience Program Curriculum Program Faculty Evaluation and Feedback Training Environment Personal Wellness Quality Improvements Communication and Patient Perceptions Other areas….(anything we missed)
47. GME Annual House Staff Survey Provides a level of detail not available from the ACGME survey Such as on call room issues, fear/intimidation issues, harassment issues, faculty Allows for text answers, comments Warning: Don’t do a survey unless you are willing to accept and act on the input
48. GME Annual House Staff Survey
49. GME Annual House Staff Survey
50. Duty Hour Reports
51. DIO’s Role Continuous monitoring of all programs Look at all of the data Take to GMEC
52. Tools ACGME surveys GME house staff survey APRs Duty hour reports ACGME letters Internal reviews Institutional report card
53. “The Report Card”
54. Site Visit (or Self Study)Preparation Ongoing Best done continuously Recommend not “last minute” Takes time and resources Needs your best “detail oriented” staff
55. Where to start?
56. Internal Scheduling/Tracking
57. Plan ahead Look at the ACGME schedule Monitor annually APRs Monitor annually program policies Use the ACGME survey data!! Use resident/program director feedback Update PIFs annually/monitor annual updates
58. Truly A Juggling Act!
59. Teach….Teach… Monthly program director & coordinator sessions Reach out to programs Assign an education specialist
60. Faculty development for program directors (& coordinators) Monthly 1 hour meetings Site visit preparation PIF writing
61. GME: Faculty Development - AY 2011-2012
62. Faculty Development Workshops Fundamentals of curriculum writing Evaluation Policies Feedback
63. How We Teach Preparing for a site visit

64. Accreditation & Maintenance of Program Quality: A Continuous Process

    Preparing for an ACGME Site Visit For Program Directors, Assistant Directors and Coordinators
65. TIMELINE: Preparing for an ACGME Site Visit Receive ACGME site visit letter 100-110 days prior to visit APR Annually Annual ACGME Resident Survey Jan to May Annual GME House Staff Survey March to May Mid-Cycle Internal Review Response to Mid-Cycle Review Midpoint between Site Visits RRC Site visit prep: Outline approach to PIF 3-4 months prior Strategize with PC/GME 3-4 months prior Submit draft to DIO 4-6 weeks prior Submit final draft to GME/DIO 3-4 weeks prior Final PIF to site visitor 2 weeks prior Meet with faculty 2-4 weeks prior Meet with residents/fellows 2-4 weeks prior
66. Site Visitor – ACGME Field Staff Two types: Field representatives and Specialist Site Visitors. (Field reps do 1940 of the 2000 annual visits e.g. 125 visits per year each – Specialist Site Visitors do approximately 60 visits) Field Representatives are employed by the ACGME, and are expected to be knowledgeable in the Program Requirements and on-site review of the medical specialty and subspecialty areas accredited by the ACGME.  Bios on ACGME web site: 31 individuals: 24 MDs, 6 PhDs, one EdD Role: produce report that verifies and clarifies the information provided in the PIF and from the ACGME resident survey Will interview program director, faculty, residents/fellows, and DIO SITE VISITOR IS A FACT-FINDER for the RRC, NOT THE DECISION-MAKER
67. Prerequisites for Accreditation I. Institutional Requirements II. Common Program Requirements III. Specialty-Specific Program Requirements
68. Read your requirements…
69. Understanding Requirements Must: Present without fail Should: Absence justified by sound rationale Desirable: Viewed as enhancement to program
70. Understanding Requirements Must: Duty hours must be limited to 80 hours per week, averaged over a 4-week period. 2) Must/Should: Adequate time for rest and personal activities must be provided. This must consist of an 8-hour time period and between duty periods and should consist of a 10-hour time period between duty hour periods.
71. Understanding Requirements Old common requirements: Assessment should include …. evaluation by faculty, patients, peers, self, and other professional staff. New common requirements: The program must… use multiple evaluators (e.g., faculty, peers, patients, self, and other professional staff);
72. Know your requirements…
73. Read your PIF…
74. Program Information Forms (PIF) Common PIF Accreditation information Sites Faculty Resident/Fellow appointments Evaluation Duty Hours
75. Program Information Forms (PIF) Specialty Specific PIF Section for each competency Duration and scope of education General institutional information Physician wellness Faculty supervision and research projects Duty hours and work conditions Best practices Comments Goals and objectives
76. PIF Strategy for Program Directors Download the PIF a year in advance Start working on one section at a time Do not expect your coordinator, other faculty or your mother to write the PIF You know your program the best of anyone!
77. Program Information Form Preparation (PIF) Review old PIF, internal review report, ACGME letters and responses (if any) Cannot just update your old PIF PIF format has changed! Prepare PIF with current program requirements open side by side!
78. Who are we? The Stanford University Programs sponsored by Stanford Hospital and Clinics
79. How do we explain ourselves? A master affiliation agreement exists between Stanford University School of Medicine and Stanford Hospital and Clinics. LPCH has also signed this agreement but they do not have their own programs
80. Master Affiliation Agreements GME Department renews annually with major affiliates VAH SCVMC UCSF Kaiser
81. Program Letters of Agreement (PLAs) Formerly educational agreements Program responsibility Needed between program and affiliate site director Must have program competency-based goals and objectives Must be signed and dated
82. Single Site Sponsor Does not apply to us! We have 87+ programs!
83. PIF Preparation Summary Write the PIF Clear, concise, in order, no missing information Beware of cut and paste errors – can not always use cut and paste tool Follow instructions! Third parties to proof Must be accurate and concordant with resident feedback to visitor Agreements are current, signed and available Faculty CVs current, especially scholarly activity Coordinate with your DIO (who must sign) as to when (and where) to mail the PIF
84. Common Errors Document not spell-checked Questions not answered Blanks (Medical School, License Expiration, etc.) Statistics not added correctly % of time for faculty inconsistent within PIF Pages not numbered Type font does not match CVs list more than requested years of publications
85. The Questions!!! Be sure to answer the question not market your program
86. Remember Answer the question concisely Only provide documentation requested Check and double check the PIF Have someone else who knows your program read the PIF
87. Finally A messy or incomplete PIF can lead to a disaster Some of the site visitors are non-MDs and some are MDs from other specialties.
88. The PIF is Done, Now What? Preparing the Files, Forms, Policies, and People Preparation before the big day The day of the visit ORGANIZATION!!!
89. Preparation for the Day Site Visitor Letter from ACGME Make contact immediately Be very accommodating Site Visitor will make specific scheduling requests Who should attend Timeline for the day Additional information that should be provided Offer: Help with travel plans Parking at SHC Detailed directions and location information Breakfast/coffee/lunch
90. Preparation for the Day Schedule Date is not usually flexible All people requested by Site Visitor must be present Inform him/her immediately if someone is unavailable Clear, precise, organized – email to Site Visitor Once it’s set don’t change…especially on the day of the visit Schedule one conference room for entire day Arrange for tours of key facilities as requested by the site visitor
91. Site Visit Preparation Logistics Schedule Chair, Faculty, Residents/Fellows (peer selected), Administrators (including DIO) Review resident/fellow files (internal audit): application, educational plans (goals and objectives), rotation schedule and call schedules, conference attendance, evaluations Group meetings: Key faculty: review program requirements and PIF Residents/fellows: Review program requirements and how you are meeting them. Review ACGME and GME resident surveys if available
92. Preparing the People Residents/fellows and participating faculty Distribute the PIF – should be read several times prior to the visit Review: ACGME Resident Survey if available (4 or >) Policies Evaluation process ACGME competencies Goals and objectives Etc. Prepare everyone, but don’t tell them what to say!
93. Engaging Residents and Fellows Site visitor will want to meet with 10-12 (all if small program) Must be peer selected, with representation from all years Review requirements, citations, surveys; fatigue/sleep deprivation education; duty hours guidelines; competencies; rotation goals and objectives; evaluation of residents, faculty, and program, and core competencies!
94. Engaging Residents and Fellows “What would you like to see improved?” Consider mock session with other PD/DIO/GME Resident Representative Arrange coverage, exclude those post call, hand off pagers Site visit is not optimal time to have new grievances aired for first time!
95. Preparation for the Day Preparing the Documentation Two types: Resident/Clinical Fellow personnel files Program Documentation Organized No loose paper (binders or folders) Be able to find things quickly when requested Make sure all documentation matches what was reported in the PIF! Prepare one location with all required documents for review
96. Preparation for the Day Program Documentation Binders and dividers Documentation of Internal Review (not full report) Program educational goals Competency-based goals and objectives (by rotation and PGY) Program Letters of Agreement (PLAs) Conference schedules and attendance Duty hour reports-Compliance Summary ONLY PIF Program policies, including: Supervision Duty hours Moonlighting Etc. Evaluations (templates) Of trainees for each rotation Semi-annual of trainee by Program Director Summative evaluations of trainee completing program Of faculty by trainees Of program by trainees Of program by faculty Rotation and call schedules Faculty CVs
97. Preparation for the Day Resident/Clinical Fellow Files Personnel files – one file per trainee Can organize by year, type of document, or both Transfers: include copies of correspondence between Program Directors Include: Appointment paperwork (each year) Evaluations of trainee Divided by competency and complete set Final evaluation for program graduates 360 evaluations 6-month evaluations by Program Director Other educational information, including: Procedure logs PowerPoint presentations In-service exam scores Conference attendance Rotation schedules Published articles and other scholarly activities Records of disciplinary actions Moonlighting records
98. Day of Visit Conference room Private and quiet Make sure room is clean Sign on door Refreshments Computer and phone access if possible All binders and files should be in room, ready upon request (back table) Page/email reminders for all participants 1 week prior, day before, and/or morning of Site Visit Program Coordinator should participate Additional help for Coordinator to serve as a “runner” 3 copies of PIF (exactly what was sent) Stick to schedule – be on time Again, be as accommodating as possible
99. 1-2 days Prior Give everyone a copy of the schedule Review files with coordinator Ensure the residency office, residency conference room, sleep rooms, are clean and orderly Prepare the conference room Do not disturb sign
100. Site Visit Day Very early – Check conference room and ensure all notebooks are in place Text page everyone 1-2 hours prior Additional staff standing by Have everything at fingertips/credenza behind table (15 second rule) 3 copies of PIF (identical to that sent) Resident files Other binders, other information
101. The RRC Decision Letter sent within 90 days of the RRC meeting where your PIF and Site Visit Findings are reviewed. Citations/areas of noncompliance delineated Accreditation action Provisional (new application) Continued full accreditation Adverse decisions (subject to appeal): Probation (proposed) Withdraw (proposed) Withhold (proposed)
102. Progress Reports and Letters RRC may request a response to major concerns Use tone of program improvement, not defensive Must be reviewed by GMEC and DIO, sent in triplicate, sent on time Only send responses to RRC when requested Written response to other concerns sent to GMEC
103. What can Programs Do Between Cycles to Improve? Annual program review and follow-up plans for improvement (a “must”) Internal review and follow-up recommendations (also a “must”) GMEC meetings, program coordinator meetings, workshops Program Directors’ Annual Retreat (Sponsored by GME) National meetings
104. Summary PLAN: Allocate enough time Call DIO/GME with any questions Learn (and have others learn): The Institutional, Common and Program-specific requirements The Core Competencies Write the PIF side by side with the requirements Anticipate the unanticipated during the site visit Engage residents/fellows in the process early
105. Lessons from a Dinosaur…. The “typical” site visit How to answer site visitor questions….. How to evaluate the site visitor And the inevitable…..”horror” stories
106. Gentle Words of Wisdom “I’m only a good teacher because I have made all of the mistakes before…myself.” Norman Shumway, M.D.
107. The Site Visitor Reach out via email to the site visitor to welcome Ask if they have any special requests Announce your availability
108. The Site Visitor Answer the questions Be terse Be tight Avoid “we used to …” “we will”
109. Gentle Words of Wisdom Keep notes Likes and dislikes of site visitors Who did what programs Any correlation between site visitors and accreditation cycle Don’t hesitate to call ACGME with problems!!!!
110. Horror Stories….. Oops…sent the wrong version of the PIF Oops…forgot to download the correct version of the PIF Oops…the Golf Course And the lavender dress vs. the golden retriever
111. Questions And that’s all folks…
112. Enjoy!
113. Questions? Ann M. Dohn adohn1@stanford.edu
114. DIOs Role in Teaching PIFMANSHIP for Program directors Coordinators
115. How We Teach PIF writing
116. PIFmanship 101 “I see a Site Visit coming in your future…”
117. PIF Writing 101 Gentle words of wisdom… The first impression of your program for the site visitor will be your PIF. Be accurate Answer the questions (be terse…be tight) Have documentation to back up your answers Start early-it takes months to write a good PIF This is not something your mother, spouse, best friend, or admin asst. can do for you!
118. General Information Glossary of ACGME terms : http://www.acgme.org/acWebsite/about/ab_ACGMEglossary.pdf Examples of “common information” required in the PIFs we provide to our Program Directors & Coordinators: 1) Hospital Statistics: 613 Licensed Beds at SHC 272 Beds at LPCH 52 Obstetric 220 Pediatric
119. 2. Major Participating Institutions (Affiliates) Children's Health Council [058191] Kaiser Permanente Medical Center (Santa Clara) [050571] Kaiser Santa Teresa [058092] Lucile Salter Packard Children's Hospital at Stanford [050572] San Mateo Medical Center [050585] Santa Clara Valley Medical Center [050438] Veterans Affairs Palo Alto Health Care System [050273]
120. 3. Library Resources http://lane.stanford.edu/services/resourcecore.html
121. Background There are actually 2 PIFS to complete: The Common PIF Addresses the program’s compliance with the Common Program Requirements common to all GME programs. The common PIF is electronically generated through the Web Accreditation Data System (WebADS) The Specialty PIF Addresses compliance with the specialty specific program requirements Word document downloaded from the ACGME Site
122. Common PIF Common Program Information Form www.acgme.org Use your ACGME assigned User ID & password Go to the PIF preparation section (left hand menu) Update the Common PIF data. Most of the data should be updated annually by your coordinator. The common PIF contains questions regarding participating sites, faculty CVs, evaluation, and duty hours. Some information is entered only at site visit time. Once all of the data is entered and VALIDATED Print
123. Common PIF –1
124. Common PIF Data feeds from ACGME WebADS TWO Original Signatures Required Program Director DIO PIF PIF
125. Common PIF – 2
126. Common PIF Respond to previous citations Data feeds from ACGME WebADS If you had a progress report due, the data remains in WebADS Automatically populated into your next PIF WebADS
127. Common PIF Major Changes are: Only since last site visit Program Format, e.g., Have you gone from 3 to 4 years? Change in resident complement? Change in Program Leadership? Change in participating sites?
128. Common PIF – 3
129. Common PIF Participating Sites Feeds from ACGME WebADS Stanford is NOT a single limited site sponsor Only required sites appear on the PIF Required means all residents rotate to that site Length of rotation Make sure the length of rotations adds up to 12 months per year across all sites
130. Common PIF - 4
131. Common PIF Participating Sites Program Letters of Agreement (PLAs) We provide a PLA template on our GME Website for Programs to use and educate that at Stanford, e.g., a separate PLA is required for the Children’s Hospital, because the Faculty is Stanford University-i.e.., The PLA may be between you and yourself…
132. Common PIF - 5
133. Common PIF Program Director MUST be able to approve the teaching faculty The Program Director MUST evaluate the faculty and approve continuation as teaching faculty Program Directors MUST comply with the university’s or medical center’s policies on selection, evaluation, promotion, disciplinary action and supervision. e.g., at Stanford we refer them to: The GME House staff Policies and Procedures – on the GME Website http://med.stanford.edu/gme/policy/
134. Common PIF The Program Director MUST be familiar with ACGME Policies and Procedures Institutional Requirements Common Requirements Program Specific Requirements
135. Common PIF Physician Faculty Roster List Alphabetically and by Site Faculty who spend at least 10 hrs. per week in resident education Make sure the data matches their CVs
136. Common PIF Faculty Teaching Staff Hours/week devoted to education Should be realistic Board certification If double boarded include both boards If you are sub-specialty program director or faculty, ensure primary boards are included
137. Common PIF- 6
138. Common PIF Potential Area for a HUGE number of citations for Programs: Must be totally completed (no BLANKS) Ensure certification is valid Ensure license is current and hasn’t lapsed
139. Common PIF Concise Summary of Role in Program: <Program Director> oversees the operations of the entire program; supervises trainees during patient-care activities, mentors trainees’ research/projects, provides professional training, tutorials (lectures and seminars); monitors duty hour compliance; coordinates evaluation of courses, rotations, trainees and faculty.
140. Common PIF Selected Bibliography Strict limit of 10 Strict limit only the last five years -“Publications from the last 5 years” excludes any before 2006… Publications should not be “in press” if submitted many years ago…. If not ABMS certified, explain…
141. Common PIF - 7
142. Common PIF - 8
143. Common PIF Number of ACGME Approved Positions Note: If you have a resident making up a maternity leave, you must ask ACGME for an extra slot if you are over your cap Actively enrolled residents Other than interns, everyone should have prior years of GME
144. Common PIF - 9
145. Common PIF-10
146. Common PIF Transferred, Withdrawn or Dismissed Residents Residents who resign are NOT dismissed Must have summative evaluation on transferred trainees Evaluations Yes residents are evaluated following each learning experience (i.e.. rotation) Yes- evaluations are documented and state how Electronically Files
147. Common PIF Methods of Evaluation Assessment Method Direct Observation Simulation OSCE Etc. Evaluator Faculty Nurses Program Director 360 (required as of 2007)
148. Common PIF -11
149. Common PIFEvaluation Describe how evaluators are educated to use the assessment methods listed above so that residents are evaluated fairly and consistently. The Program Director meets with evaluators annually before new fellows start in the training program and reviews, answers questions and discusses the core competencies and core competency performance evaluations used in the program and assessment methods with the faculty. The electronic assessment system and the scales and their use is also reviewed and discussed in faculty meetings in order to ensure that faculty are fully educated and up to date with the assessment methods and processes.
150. Common PIFEvaluation Describe how residents are informed of the performance criteria on which they will be evaluated. At the beginning of each academic year, the Program Director conducts an orientation to address several key issues related to the residency including the performance criteria on which the residents will be evaluated. During this orientation session, the Program Director carefully details the specific evaluation methods to be used. Both the criteria and methods are also documented in the residency handbook. Additionally, at the beginning of each block rotation, the rotation director meets with the resident and carefully delineates the expectations and performance criteria on which the resident will be evaluated for that specific rotation.
151. Common PIFEvaluation Describe the system to ensure that faculty complete written evaluations of residents in a timely manner following each rotation or educational experience. At the end of each block rotation, our online resident data collection and tracking system, MedHub, sends an automated reminder to the service attending(s) to evaluate the resident(s) on that particular rotation. Using MedHub, the Program Coordinator tracks pending evaluations and follows up with faculty as needed to urge them to complete their evaluations on time. In the unlikely event that a faculty member is unresponsive to the coordinator's requests to complete evaluations, the Program Director contacts the faculty member and requests him/her to complete the evaluations.
152. Common PIFEvaluation Describe the process used to complete and document written semiannual resident evaluations, including the mechanism for reviewing results (e.g., who meets with the residents and how the results are documented in resident files). Residence performance is evaluated by the teaching faculty at the conclusion of each rotation using an electronic evaluation form (MedHub). Additionally, each faculty mentor meets with their resident advisee quarterly and documents a summary of the meeting to be placed in the resident’s file. The Program Director meets with each resident on an individual basis semiannually to review the accumulated performance evaluations and mentor notes, provide feedback, and update the resident’s learning plan as appropriate. A summary of these meetings is documented and placed in the resident’s file. The residents are free to review the contents of their records at any time.
153. Common PIFEvaluation Describe the system used by the residents to provide annual confidential written evaluations of the teaching faculty (have examples and forms available for review by site visitor). Residents annually confidentially evaluate the teaching faculty using <system>. The electronically submitted <system>. evaluation forms are anonymous. Paper copies of completed evaluations without personal identifiers are printed by the Program Coordinator, reviewed by the Program Director, and presented to individual faculty members for review and consideration. Written evaluations are completed electronically and anonymously by using a centralized system used throughout the hospital to provide annual aggregated data. Fellows are sent e-mail reminders about completing evaluations in a timely manner by the Fellowship Coordinator. The program coordinator follows up, aggregates the results and forwards them to the Program Director. The Program Director reviews the results and takes any actions appropriate with individual program faculty.
154. Common PIFEvaluation Describe the program's (or Department's, if applicable) system for evaluating and providing feedback to the teaching faculty. Residents annually evaluate the teaching faculty using the <system>. The electronically submitted evaluation forms are anonymous. Paper copies of completed evaluations without personal identifiers are printed by the Program Coordinator, reviewed by the Program Director, and presented to individual faculty members for review and consideration. If and when the evaluations reveal particular issues with the teaching faculty, the Program Director meets with the particular faculty more urgently to address those issues and concerns. Additionally, the Program Director meets with the all faculty on an annual basis to review resident feedback and to effect any necessary changes.
155. Common PIFProgram Evaluation Describe the approach used for program evaluation. Residents and faculty annually evaluate the program using the <system>. The electronically submitted evaluation forms are anonymous. Residents and faculty are also encouraged to provide feedback to the Program Director whenever any issue arises or as they see opportunities for improvement. Additionally, residents and faculty participate in an Annual Program Improvement Meeting led by the Program Director. Aggregated data including the most recent ACGME survey results and the resident/faculty program evaluation results are reviewed and used at this meeting to improve the program. The Program Coordinator keeps minutes during this annual meeting and documents any plans to address areas requiring improvement. Action plan progress is monitored and documented by the Program Director.
156. Common PIFProgram Evaluation Describe one example how the program used the aggregated results of residents' performance and/or other program evaluation results to improve the program (have the written plan of action available for review by the site visitor). Annual fellowship retreats including fellows and faculty are conducted each year in May or June (Whenever yours is held) ….the agenda for these meetings includes review of all the aforementioned aggregated data as well an open discussion about educational quality and compliance with the program requirements and suggestions for improving the program for incoming residents. (See GME Website for more information on this and examples of written action plans) http://med.stanford.edu/gme/program_directors/eval_prog.html
157. Common PIF -12
158. Common PIF Describe the improvement efforts currently undertaken based on feedback from the ACGME Resident Survey. You MUST review your ACGME Annual Survey each year, bring the results to your Annual Program Retreat, address each area of concern (more than 20%) or any Duty Hour negative response
159. Common PIF DUTY HOURS Use the summary from your tracking system reporting function Describe
160. Common PIF Briefly describe how the faculty provides appropriate supervision of residents in patient care activities. Faculty discusses all patient care history and physical findings with resident. Faculty then confirms both the history and physical findings of the patient. The patient is then discussed in detail, education given regarding the patient illness or complaint, education materials are provided to the resident and finally, a care plan is developed and provided by the resident and confirmed by the faculty.
161. Common PIF Other Questions that may be included in you Common or Specialty PIF
162. Other PIF Questions Skills and Competencies Describe how residents are informed about their assignments and duties during the residency. This should be in writing and verbal. For example, all fellows are given the program training manual which describes their assignments and duties during orientation to the program. Each fellow signs an attestation that they have received and read the manual. The manual is discussed during orientation with the fellows as a group.
163. Other PIF Questions GENERALIZE- Grievance Procedures Describe how the program handles complaints or concerns the residents raise. The program follows the Dispute Resolution (grievance ) procedures described in the Stanford University Medical Center House Staff Policies & Procedures, pages 20-24, distributed to residents at the beginning of their residency and available on the GME website
164. Other PIF Questions Describe how the program monitors for excessive service and modifies the program accordingly. The residency has several methods to monitor for excessive service and modify the program if required: Formal rotation evaluations with alerts: Residents complete monthly evaluations using the system that are reviewed and communicated with the rotation directors. Committee on Residency Training and Clinical Services: Each class has resident representatives meet with the Program Director monthly for a formal agenda to review and address issues in the residency program. The rotation directors are sent the residents’ monthly feedback on a quarterly basis (to ensure confidentiality for the residents’ feedback) Yearly, the residents have a retreat in which each aspect of the program is reviewed. Area of concern are brought back to the Committee on Residency Training. The ACGME and GME resident survey questions with respect to excessive service are reviewed. Informal feedback: the residents are encouraged to let the Chief Residents, the Faculty, and the Program Director know immediately when either the service load or the educational activities are compromised. Weekly meetings of the program director and chief residents to discuss the residents and experiences on individual rotations. Computerized entry and monitoring of all hours is performed in MedHub.
165. SPECIALTY PIF The specialty specific PIF is located on the ACGME website under your Residency Review Committee. http://acgme.org/acWebsite/navPages/nav_PDcoord.asp The specialty PIF contains the questions regarding the ACGME core competencies, block diagrams for your program, a narrative description of the program, scholarly activity, and often case logs.
166. Specialty PIF-Practice-Based Learning & Improvement Describe one learning activity in which residents engage to identify strengths, deficiencies, and limits in their knowledge and expertise (self-reflection and self-assessment); set learning and improvement goals; identify and perform appropriate learning activities to achieve self-identified goals (life-long learning). Programs may use a structured process for reflection in which a faculty advisor guides the resident in using feedback and evaluations to inform the self assessment process. Documentation of the semi-annual evaluation meetings in which this process is demonstrated would provide evidence that this requirement is being addressed.
167. Specialty PIF-Practice-Based Learning & Improvement Describe one example of a learning activity in which residents engage to develop the skills needed to use information technology to locate, appraise, and assimilate evidence from scientific studies and apply it to their patients’ health problems. The description should include: a) locating information, b) using information technology, c) appraising information, d) assimilating evidence information (from scientific studies), e) applying information to patient care. An appropriate learning activity could be a structured activity such as a journal club presentation, critical appraisal of a topic, or educational prescription with appropriate faculty oversight and formal assessment of skills. Additional documentation would be the written goals and objectives for this learning activity and how residents are assessed.
168. Specialty PIF-Professionalism Describe at least one learning activity, other than lecture, by which residents develop a commitment to carrying out professional responsibilities and an adherence to ethical principles. This activity should be structured, should demonstrate active faculty involvement (not just passive role modeling) and timely feedback to residents, and should include a mechanism for collecting evaluations (including routine multi-source assessment).
169. Specialty Systems-Based Practice Describe an activity that fulfills the requirement for experiential learning in identifying system errors. Important elements may include identified faculty to guide the activity, mechanism to ensure active engagement by each resident, and evidence of experiential learning (not just passive presence at conferences and meetings) in which residents participate in identifying a system problem or error and contribute to a potential solution. Additional documentation could include written goals and objectives for this learning activity and how residents are assessed. Aggregated resident outcomes may be in the form of percentage of residents that completed a patient safety or other Systems-Based Practice project by the end of training, annual list of improvements that resulted from such projects, etc.
170. PIF UPDATE-REVISEBriefly describe how the faculty provides appropriate supervision of residents in patient care activities. Faculty discusses all patient care history and physical findings with resident. Faculty then confirms both the history and physical findings of the patient. The patient is then discussed in detail, education given regarding the patient illness or complaint, education materials are provided to the resident and finally, a care plan is developed and provided by the resident and confirmed by the faculty. Face-to-face feedback regarding patient care is given as quarterly evaluation by all faculty.
171. Table of Contents
172. Common Errors Table of Contents Inaccurate Pages not numbered or not numbered correctly Document not spell-checked Question asked not answered Statistics not added correctly % of time for faculty inconsistent within the PIF Type font does not match
173. Putting It All Together REVISE---One complete PIF (common and specialty) is due to the site visitor at least 14 days before the site visit. Stanford’s GME needs the complete PIF 14-30 days before it is send to the site visitor. (the math means that you should have the completed document ready 6 weeks BEFORE your site visit) Stanford’s GME usually has 3 site visits during a week. This translates to 3 PIFS to review. Help the poor folks out….and be EARLY!!!! The more time you give GME…the more time for editing, comments, and changes. You really cannot have too many eyes look at one PIF!!
174. Save time in the PIF Site Visit/Internal Review Process Leverage Inexpensive Technology Applications e.g., PDF – PIF Review Functionality Microsoft Word Track Changes Documents revisions that are required Electronic copies can easily be uploaded to internal web sites for easy access and documentation Enables team reviews – from “anywhere” Provides Program Directors easy review of recommendations Provides Program Coordinators a streamlined way to make many of the required changes
175. The Land Mines As you write your PIF… remember to read the question carefully. One example does NOT mean several “How” does not mean “we do” No “will’s” No “see attached – “see below”