Ontario Laboratory Accreditation Peer Assessment Process Peer Assessment Visits: What to expect

1. Ontario Laboratory AccreditationPeer Assessment ProcessPeer Assessment Visits: What to expect

2. Purpose of a Peer Assessment • To determine • Conformance to OLA requirements (based on ISO 15189, generally accepted principles of good practice and applicable legislation) • Conformance to testing scope specified on licence • To (ultimately) issue accreditation certificates

3. The Value of Peer Assessment • Ensure that laboratories meet mandatory quality management system criteria, in order to give formal recognition that the laboratory is competent to carry out examinations • Assessment by peers ensures consideration is given to new technical developments, unique challenges

4. Relationship between OLA and EQA OLA ensures that quality management processes are in place while EQA monitors if processes are effective (output) • Continued OLA accreditation dependent on demonstration of ongoing competence in EQA surveys • EQA (targeted) on-site consultations will continue

5. Peer Assessments • Scheduled once every five years* • Notification 120 days in advance of expected visit • Mutually agreeable date set • Assessment against essential requirements only * May occur more frequently

6. Important Notes Laboratory to identify a site coordinator who works with the OLA staff coordinator regarding visit arrangements A pre-assessment visit questionnaire and self-assessment of the quality manual (along with the quality manual) are requested before the visit

7. The Peer Assessment Process Steps in a peer assessment: • Notification of laboratory • Visit coordination and team assembly • Confirmation of team and agenda • Assembly of assessor packages • On-site assessment visit (opening and summation meeting) • Formal summary report • Implementation and submission of corrective actions • Review of corrective actions • Certificate Issue

8. Multi-Site Assessments • Criteria • Single management structure • Single QMS • Single assessment team • Sites within reasonably close proximity • Assessment(s) occur within a week

9. Multi-Site Assessments • To corporately-linked facilities conducted as a single visit • One opening meeting • Assess management and QMS only once • Confirm selected issues at each site • One summation meeting

10. Assessor Binders Assessors receive the following information prior to visits: • Itinerary, travel details and applicable tickets/documents • Team contact list • Visit agenda • Laboratory testing scope, staff list, equipment list • EQA participation and performance information • Results of prior self-assessment • Completed pre-visit questionnaire • Summary of quality manual review and quality manual • Checklists • Procedures • Evaluation form • Expense form

11. Logistics of Arranging the Visit Priority “To-do’s”: • Once the designated correspondent and official laboratory director are notified of upcoming peer assessment visit by mail and telephone, a laboratory site coordinator must be identified to work with the OLA staff coordinator. • Laboratory management must ensure that all personnel are aware of the upcoming peer assessment visit. • Laboratory management must ensure that all personnel are familiar with OLA requirements.

12. Logistics of Arranging the Visit, cont. • Once the designated correspondent and official laboratory director are notified of upcoming peer assessment visit by mail and telephone, a laboratory site coordinator must be identified to work with the OLA staff coordinator. • Complete and return pre-visit questionnaire • Complete QMS self-assessment and submit with Quality Manual • Review and approve proposed agenda • Facilitate team confirmation • Book conference rooms, audiovisual equipment (if applicable) • Arrange lunch for assessors • Ensure appropriate staff are present for opening and summation meetings • Provide lab coats and visitor badges (if applicable) • Arrange for 10 minute interviews with two clients (e.g. physician, nurse)

13. Logistics of Arranging the Visit, cont. Priority “To-do’s”: • Laboratory management must ensure that all personnel are aware of the upcoming peer assessment visit. • Notify key personnel in other departments, if applicable(e.g. safety officer, LIS specialist, materials management, engineering, human resources, POCT specialist) • Review purpose of assessment

14. Logistics of Arranging the Visit, cont. Priority “To-Do’s”: • Laboratory management must ensure that all personnel are familiar with OLA requirements. • Review OLA requirements • Review “what to look for” discipline information • Review list of documents and records for review during visit

15. Opening Meeting With Facility Staff • Team leader conducts: • Introductions • Explains assessment process • Confirms agenda, personnel and arrangements • Explains evaluation feedback mechanism

16. Assessment: Gathering Information • Three primary techniques: • Observation: observe activities • Interviewing: asking questions using the checklist • Reviewing documents and records: • Polices, processes, procedures • Records • Each assessor has his/her own style – but all will use a combination of these techniques

17. Practical Tips • Preparation in advance is CRITICAL • Be familiar with requirements and “what to look for” • Be aware of what assessors might ask for • Produce available evidence of conformance • Assessors will share their findings, but may need to confirm with the team to determine if a non-conformance is major or minor (“Post-it” notes used by assessors do not necessarily mean non-conformances)

18. Major Non-conformance Non-fulfillment of any requirement to a major degree: • Requirement not addressed by laboratory’s quality manual or operating procedures • Procedures are consistently NOT followed • Existing protocols fail to address requirement • Non-conformance directly impacts patient safety • Consistent/persistent incidence of non-conformance • Repeated incidence of non-conformance in the majority of sections of the laboratory

19. Minor Non-Conformance Non-fulfillment of any requirement to a minor degree: • Isolated incident of non-conformance • Adherence to procedures is inconsistent (usually followed but sometimes not) • Existing protocols address requirement but are not necessarily followed

20. The “Numbers Game” • Certificates are NOT issued based on the number of non-conformances observed • Certificates are issued based on your response in 90 days (how non-conformances are addressed through corrective action) • A single issue can generate more than one non-conformance

21. Quick-Fixes on Site • A laboratory may “FIX” a non-conformance while the team is present, at the discretion of the team leader provided that: • Document control is in place • Approval is indicated on the new documentation • New policy/process/procedure requires absolutely NO staff training, information or orientation • Clear evidence of acceptable practice exists • Sole motivation is not to reduce # of non-conformances

22. Conflict Resolution • Encourage staff to provide evidence of conformance prior to the summation meeting • When a disagreement occurs, team leader mediates • There is a mechanism to submit an “Errors of fact” following the assessment - welcomed by QMP-LS within two weeks after receipt of printed summary report - not to be confused with a challenge to the text of a requirement

23. Pre-Summation/Summation Meeting Lead by team leader: • All non-conformances conveyed • Verbal summary ONLY • Pre-summation meeting may occur with key staff • Format of summation varies: • Dependent on the number and nature of non-conformances • Preference of laboratory management • Follow-up process explained

24. Following the Visit • Evaluation feedback to QMP–LS • Printed summary report issued within 30 days • “Errors of fact” must be submitted within 10 business days of receipt

25. Following the Visit, continued • Corrective action plan due within 90 days from visit • Be specific, provide documentation • Corrective action reviewed by OLA staff coordinator, team leader and Advisory Panel • Further evidence or clarification may be requested

26. Certificate Requirements • Evidence provided that: • Majors: corrected or suitably addressed • Minors: satisfactory action plan in place = 5 year Accreditation • If Majors cannot be corrected but satisfactory action plan is demonstrated = 2 year certificate with opportunity to upgrade to a 5 year certificate when action plan completed

27. Accreditation Certificates • Specific to facility licence and identified scope • Laboratory name and address specified • Applicable scope of testing attached • Formal recognition of competence in meeting ISO 15189, generally accepted principles of good practice and applicable legislation • Conditional upon satisfactory performance in EQA

28. Summary Laboratories know prior to peer assessment visit: • When visit will occur • Who assessors will be • What will be assessed Laboratories are expected to: • Be prepared for visit • Be familiar with requirements • Provide evidence of conformance • Submit any “Errors of fact” following visit • Provide corrective action following receipt of printed summary report: • Documented evidence • Detailed action plans