1 / 43

STRUCTURED ORAL INTERVIEW TO ASSESS PHYSICIAN’S CLINICAL COMPETENCE

STRUCTURED ORAL INTERVIEW TO ASSESS PHYSICIAN’S CLINICAL COMPETENCE. Dr. François Goulet Dr. Marguerite Dupré Dr. André Jacques Practice Enhancement Division Collège des médecins du Québec. Monitoring and Enhancement System to Improve Physician Performance. Monitoring. Enhancement.

fabiano-knight
Download Presentation

STRUCTURED ORAL INTERVIEW TO ASSESS PHYSICIAN’S CLINICAL COMPETENCE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. STRUCTURED ORAL INTERVIEW TO ASSESS PHYSICIAN’S CLINICAL COMPETENCE Dr. François Goulet Dr. Marguerite Dupré Dr. André Jacques Practice Enhancement Division Collège des médecins du Québec

  2. Monitoring and Enhancement System to Improve Physician Performance Monitoring Enhancement Step 3 Detailed Needs Assessment Remediation Programs Monitoring Enhancement CME Programs for Groups and /or individuals Assessment of physicians in Need / at Risk Step 2 Enhancement Monitoring Screening of all physicians Feedback to all physicians Step 1

  3. The Structured Oral InterviewNeeds Assessmentand competence evaluation • For enhancement CME • Reorientation • Back to practice • Personal needs • For remedial CME • Disciplinary Committee or Inquiry Division • Peer review program

  4. The Structured Oral Interview • Evaluation  Exam • Initially developed to assess needs and competence of physicians • Evaluation done by 2 peers • Length: Half-day to 1 day • Expectations: Practice competency judged in the main stream • No time duration required to complete each vignette

  5. The Structured Oral Interview Construction: • Realistic, representative, standardized clinical vignettes • Focus on specific aspects of clinical competence • Data gathering (questionnaire and physical exam) • Diagnosis • Investigation plan • Treatment plan • Drug prescriptions • Follow-up • Each case is built on a key-feature fashion • No cascade errors: answers are given following each section • Validation of vignettes by peers

  6. The Structured Oral Interview Scoring • Good score for each item selected • No penalty for wrong answer • Assessors evaluation report based on psychometric data and their notes • Assessors make recommendations for an educational prescription

  7. The Structured Oral Interview Psychometric Data • Face validity: candidates recognized and affirm the representativity of the clinical vignettes to their practice • Internal consistency  Cronbach 0.91 • Inter-raters reliability  91.2% answer • Main Score obtained by candidates  50.5%

  8. The Structured Oral Interview Evaluation Domains A. Interview Vignettes • Data gathering • Data interpretation • Problems solving B. Simulated Patients • Questionnaire • Physical Examination • Patient-Doctor Communication C. Script Concordance Test • Clinical reasoning

  9. The Structured Oral InterviewSpecification TableFamily Medicine • Domain assessed (20 vignettes) • Questionnaire (12) • Physical examinations (13) • Diagnosis (12) • Investigation plan (9) • Drug prescriptions (13) • Treatment plan and counselling (8) • Follow-up (6)

  10. The Structured Oral Interview EXAMPLE OF AN INTERVIEW

  11. Case # 123 Discipline: Geriatrics Starting point: Lumbago Outcome: Aortic anevrysm Complexity level: 1/2 Dimensions assessed and corresponding questions: History 1 Physical examination 2 Diagnosis 3 Management plan 4

  12. Case # 123 To be read to the candidate Mr. Hargrove, 75 years old, has come to see you without an appointment. He is a new patient. He tells you that for the last two days he has felt a heaviness in the left lumbar region, as well as an undefined discomfort in the abdomen. Q1 YOU BEGIN WITH SOME QUESTIONS. WHAT ARE YOU LOOKING FOR?

  13. Case # 123 Expected responses Tick () when mentioned by candidate A. Previous cardiac or vascular problems B. Apparition mode of the pain C. Precise site of the pain D. Permanent or intermittent pain E. Pain intensity F. Aggravating or trigger factors G. Trauma H. Urinary tract symptoms I. G-1 symptoms (nausea, vomiting or modification of stools) J. Fever or chills

  14. Case # 123 To be read to the candidate Mr. Hargrove tells you that he has been receiving treatment for hypertension for many years. He is taking a B-Blocker. He says this is the first time that he has felt this way. He has not suffered any trauma or made any bad move that would have caused this pain. He just woke up one day with this pain in the left lumbar region. The pain is dull and constant, and it seems to start in the periumbilical area and radiate through the back. He otherwise feels quite well, without any other symptoms. He does not present with nausea, vomiting, urinary problems or fever. Q2 DESCRIBE WHAT YOU ARE LOOKING FOR IN THE PHYSICIAL EXAMINATION

  15. Case # 123 Expected responses Tick () when mentioned by candidate A. Vital signs B. Cardiac auscultation C. Abdominal examination D. Abdominal aortic palpation E. Peripheral pulses F. Low back examination G. Renal punch H. Abdominal murmur

  16. Case # 123 To be read to the candidate The examination confirms that Mr. Hargrove's vital signs are normal. The cardiovascular examination is also normal, as well as the peripheral pulse. The examination of the lumbar area is normal and the renal punch is negative. There is a normal peristaltis in the abdomen and the abdominal wall is supple. There is a murmur in the area of the right renal artery and a pulsatile periumbilical mass of about 8 x 6 cm, sensitive to palpation. Q3 WHAT IS YOUR DIAGNOSIS?

  17. Case # 123 Expected responses Tick () when mentioned by candidate A. Dissequant anevrysm or near ruptured abdominal aortic anevrysm

  18. Case # 123 To be read to the candidate You conclude that his abdominal and lumbar pain is caused by an aortic anevrysm in the abdomen that may be dissecting. Q4 WHAT COURSE OF ACTION WILL YOU TAKE?

  19. Case # 123 Expected responses Tick () when mentioned by candidate A. Immediate reference to emergency room for vascular surgery

  20. The Structured Oral Interview Simulated Patients • Based on simulated standardized patients • The candidate must question or perform a physical examination • Checklist completed by an evaluator • Items on checklist are validated by peers • Scoring: • Score for each item depending on its importance • Total score for each station

  21. The Structured Oral Interview Simulated Patients 9 Simulated patients • 3 Patient-Doctor communication • Starting the interview • Counselling • Ending the interview • 6 Questionnaire or physical examination • Abdominal exam • Cognitive status questionnaire • Lumbar back examination • Neck examination (thyroid included) • Examination of legs (thrombophlebitis) • Knee examination

  22. A. Inspection - painless position B. Movement of the back Lateral movement Rotation Flexion (Schoeber) Extension C. Lasègue or tripod Check () if Q 1 Score right answer ______ A 3 pts ______ B1 4 pts ______ B2 4 pts ______ B3 4 pts ______ B4 4 pts ______ C 25 pts The Structured Oral InterviewSimulated Patients Q 1 Physical Examination If needed, ask again the candidate to describe orally what he/she is doing

  23. D. Sensation: L4 ou L5 (including space between 1st and 2nd toe) E. Muscular strength: Flexion and extension of the thigh Flexion and extension of the leg Dorsiflexion and planiflexion of the foot Check () if right answer ______ D 6 pts ______ E.1 3 pts ______ E.2 3 pts ______ E.33 pts The Structured Oral InterviewSimulated patients Q 1 Neurological examination of the leg (cont'd)

  24. F. Reflexes: Patella (L3-4) Heel (L5 -S1) G. Neurological examination of both legs Check () if right answer ______ F.1 3 pts ______ F.2 4 pts ______ G 4 pts Total 70 pts The Structured Oral InterviewSimulated patients Q 1 Neurological examination of the leg (cont'd)

  25. The Structured Oral InterviewScript concordance test • Clinical reasoning assessment and the capacity to make micro-decisions • Paper-Pen tool • Measures the level of concordance between the candidate’s decision and the modal answer of a panel of peers • Probes the organization of clinical reasoning

  26. The Structured Oral InterviewScript concordance test For diagnostic assessment

  27. The Structured Oral InterviewScript concordance test

  28. The Structured Oral InterviewScript concordance test • 64 QUESTIONS (Alpha = 0,90) • Evaluation domain • Diagnosis (questionnaire) • Diagnosis (physical exam) • Investigation plan • Treatment plan • Panel: 13 family physicians randomly selected • Table of specification

  29. The Structured Oral InterviewReport from assessorsFamily Medicine Case-by-case analysis • Performance analysis on: • Emergency cases • Diagnosis • Pharmaceutical treatments • Counseling • Follow-up • Referrals • Recommendations and educational prescription

  30. The Structured Oral Interview1995 -2005 (n=100 )

  31. The Structured Oral InterviewOrigin of candidates 1995-2005(n = 100 )

  32. TheStructured Oral Interview Poor results Excellent results Imposed clinical training program Tutorial 1/2 day/week Recommendations of formal CME No recommendations Voluntary clinical part time training program Part time 2 to 3 days/week full time Recommendations of focus readings Without limitation of practice With limitation of practice

  33. The Structured Oral Interview Educational prescription Source Reason SOI's Results Formal CME in pharmacotherapy 2 conferences over 2 years PIC Success/Comments 15/19 cases 6/6 simulated patients 19/19 treatment plans 4/5 follow-ups Good clinical skills Good insight about his limits Some hesitation in prescriptions even a good performance • General practitioner • In solo practice • x 25 years • Very poor chart • keeping skills • Concerns about quality • of his practice, • specially: • Data gathering • Management plan M 70 years

  34. The Structured Oral Interview Educational prescription Source Reason SOI's Results Success/Comments 10/18 cases 2 /6 simulated patients 14/18 treatment plans 3/5 follow-ups Good medical knowledge Unstructured clinical skills Difficulties in the questionnaire and physical examinations Good insight of his limits Imposed part-time clinical training 2 days/week x2 months without limitation of his practice PIC • General practice in • office and regional • hospital x 1979 • Incomplete • questionnaires and • physical • examinations • Poor differential • diagnosis • Incomplete • management plans M 57 years

  35. The Structured Oral Interview Educational prescription Source Reason SOI's Results • General practice • In a solo practice • x 35 years • Complaint to the • Inquiry Division: • Poor management • and treatment plans • Concerns about the • quality of his • practice Imposedclinical training x 6 months with limitation of his practice Inquiry Division Success/Comments 3/20 cases 2/5 simulated patients 6/21 treatment plans 0/5 follow-ups Poor medical knowledge Unstructured clinical skills Poor questionnaire skills M 64 years

  36. The Structured Oral Interview IT'S YOUR TURN TO PROPOSE EDUCATIONAL PRESCRIPTIONS

  37. The Structured Oral Interview Educational prescription SOI's Results Source Reason General practitioner In a solo practice since 1970 Complaint: Alcoolism Competency assessment required Inquiry Division Success/Comments 14/20 cases 4/6 simulated patients 14/21treatment plans 4/5 follow-ups Well structured clinical skills Weakness in drug prescriptions Difficulties in the questionnaire and the physical examination M 53 years

  38. The Structured Oral Interview Educational prescription SOI's Results Source Reason Imposed part-time clinical training 2 days/week x3 months without limitation of his practice General practitioner In a solo practice since 1970 Complaint: Alcoolism Competency assessment required Inquiry Division Success/Comments 14/20 cases 4/6 simulated patients 14/21treatment plans 4/5 follow-ups Well structured clinical skills Weakness in drug prescriptions Difficulties in the questionnaire and the physical examination M 53 years

  39. The Structured Oral Interview Educational prescription SOI's Results Source Reason General practitioner Voluntary limited practice to mental health in1986 From 1999, she changed her scope of practice to general practice Concerns about her quality of practice Success/Comments 6/20 cases 1/6 simulated patients 13/21 treatment plans 2/5 follow-ups Poor clinical skills No differential diagnosis Unstructured physical examinations PIC F 49 years

  40. Educational prescription Source SOI's Results Reason PIC The Structured Oral Interview General practitioner Voluntary limited practice to mental health x 1986 From 1999, she changed her scope of practice to general practice Concerns about her quality of practice Success/Comments 6/20 cases 1/6 simulated patients 13/21 treatment plans 2/5 follow-ups Poor clinical skills No differential diagnosis Unstructured physical examinations Imposed full-time clinical training x3 months with limitation of her practice F 49 years

  41. The Structured Oral Interview • Advantages • Valid and reliable evaluation • In a short time (one day) • Well accepted by candidates • No rush to complete the process • Representative of their practice • Good inter-rater reliability • Assessment of all aspects of clinical practice • Limits • Costly ($2500-$4000/candidate) • Risk of information leakage if cases bank too small • Human ressources required

  42. The Structured Oral Interview For more information: • E-mail: goulet.cmq@sympatico.ca • Tel: 514 933-4441, ext. 5237

  43. The Structured Oral Interview QUESTIONS?

More Related