Reliability Assurance Initiative

1. SPP Compliance Forum May 23, 2013 Reliability Assurance Initiative Thomas P. Tierney Director of Compliance, MRO

2. Common Mission Improve the Reliability of the Bulk Power System

3. Improvement Is the Goal • We have very reliable systems within MRO/SPP, but we can still improve by identifying problems and fixing them – no weak links • There is always opportunity for improvement within the design criteria of an interconnected system

4. Demystifying Internal Controls No, Really… What Is an Internal Control?

5. Nothing New • Registered Entities have been managing reliability for decades – they have management practices (i.e. controls) around reliability • Existing practices have been translated into the Reliability Standards and documented – “operationalizing compliance” • Don’t overthink “internal controls”

6. Definitions • Risk • Possibility that something undesirable will happen • Measured as a combination of likelihood and impact • Control/Control Activity • Policy, procedure, checklist, etc. designed to minimize the opportunity for a risk to be realized • Internal Control • Control activity performed internally, not by a third party • Management practices that include control activities performed internally (“self monitoring”)

7. Types of Risk • Inherent Risk • Risks “built-in” to a given entity, based on geography, what facilities it operates, “interconnectedness,” etc. • Reliability Standards are designed to mitigate inherent risk in a broad sense • Control Risk • Risk that management practices or control activities are not achieving their reliability or compliance objectives • Detection Risk • Risk that possible violations are going unnoticed • Residual Risk • Risk that remains after application of a control and other mitigating factors • Difficult and expensive to eliminate 100% of risk – we must live with some risk

8. Types of Controls • Preventive • Controls designed to stop something from occurring • Detective • Controls designed to identify when a possible violation has occurred and facilitate timely remediation • Also known as “Monitoring” controls

9. Control Hierarchies • Multiple, complementary controls that work together to reduce risk (“Defense in depth”) • Primary • Secondary • Tertiary • Secondary and Tertiary controls serve as a “safety net” in case the Primary control does not function as expected • Each subsequent tier of controls further reduces residual risk

10. Examples • Protection System Maintenance and Testing • Relay technicians complete work orders according to a pre-defined checklist to prevent steps being skipped or performed incorrectly • Supervisors review and approve completed work orders to verify technicians’ proper use of the checklist • A sample of work orders is reviewed by Internal Audit to verify accuracy and completeness

11. Examples Procedure/ Process Control Control Activity Control Type Checklist followed and completed, exceptions noted, follow-up notes signed Program Documents (Procedures) StandardWork Order Primary Control Review for completeness and accuracy, follow-up actions closed or scheduled to be completed, signed Secondary Control Supervisory Review Periodic sampling of work orders to determine program is being completed and properly reviewed Management Oversight Tertiary Control

12. Examples • Training • Management establishes training objectives and reviews training materials to confirm objectives are met • Individuals are tested after completion of training to ensure effectiveness of delivery • Supervisors conduct performance observations to verify past training has been effective and to identify additional training needs

13. Examples Procedure/ Process Control Control Activity Control Type Management establishes training objectives and reviews training materials to confirm objectives are met Program Documents (Procedures) Training Objectives Primary Control Individuals are tested after completion of training to ensure effectiveness of delivery Secondary Control Training Evaluation Supervisors conduct performance observations to verify past training has been effective and to identify additional training needs Performance Observations Tertiary Control

14. Examples • Cybersecurity • Systems are configured to require passwords to prevent unauthorized access • All changes to systems are reviewed, approved, and tested to ensure that unauthorized changes do not occur • Periodic reviews are conducted to ensure that password controls adhere to corporate security policies

15. Examples Procedure/ Process Control Control Activity Control Type Systems are configured to require passwords to prevent unauthorized access Security Policies Password Controls Primary Control All changes to systems are reviewed, approved, and tested to ensure that unauthorized changes do not occur Secondary Control Configuration Management Procedures Configuration Management Periodic reviews are conducted to ensure that password controls adhere to corporate security policies Security Assessments Tertiary Control

16. Reliability Assurance Initiative Focusing on Risk

17. Current State • “One size fits all” compliance model • NERC Actively Monitored Standards do not change based on regional differences, entity size, etc. • No consideration of management practices (i.e. controls) around reliability standards • Zero-defect approach to enforcement is burdensome • Every violation requires a regulatory filing regardless of severity • Self-reports require significant effort • Administrative Citation Process (ACP) & Find, Fix, Track (FFT) are not sufficient • Expediting enforcement won’t solve the problem

18. Key Elements of RAI • Shape compliance monitoring and mitigation based on risk • Reserve enforcement for most significant risks

19. Scoping of Work • Assessment of each entity’s inherent risk • Some factors influencing assessment • Facilities • Special Protection Systems • IROLs • Geographic location • Functions performed • Connectivity (physical and cyber) • EMS/SCADA system • Compliance history

20. Scoping of Work • What does a risk assessment look like? • Not a letter grade or single rating • Entities will not be compared and ranked • Assessment will look more like a matrix • Certain families of standards may be higher risk for one entity, less risky for another

21. Scoping of Work • Internal controls established by each entity must be identified • Evaluation of select controls to determine effectiveness • Design – Is the control, as documented, adequate to address the risk? • Operational – Is the control implemented as designed? • Effective controls reduce residual risk to an acceptable level • MRO staff can rely on effective controls • Regulatory scope can be adjusted – less auditing and testing (or none) where strong controls exist

22. Scoping of Work • Risk assessments and internal controls will be leveraged across all compliance monitoring activities • Internal emphasis should shift over time toward maintaining effective controls around Reliability Standards • Continue to identify and correct issues in a timely fashion • Focus on reliable operations first • Compliance should be a natural outcome of strong operations

23. Compliance Exceptions • “Compliance Exceptions” represent lower risk violations • Do not represent significant risk to the BES • Identified by an entity itself or by regional staff • Initially tracked at the regional level • No enforcement proceedings, no penalties • Mitigation will always be important • What was done to address the problem itself? • What is being done to prevent recurrence?

24. Compliance Exceptions • Enforcement will focus on most significant or high-risk issues • Violation poses significant risk to reliable operation of the BES, e.g. cause or contributing factor in a cascading event • Multiple smaller issues may aggregate into a bigger problem or are indicative of a poor control environment • Willful misconduct

25. MRO Pilots • Compliance audit • Tools being developed with input from industry, the Regions, and NERC • Currently developing risk assessment • Internal controls evaluation to occur during June & July • Scope will reflect risk and presence of effective controls • Audit completion in Q4 of 2013

26. MRO Pilots • Self-certification • Transition from blanket, “check the box” approach to narrowly focused self-certifications • Scope limited to FAC-008-3 R6 based on problems identified on recent audits • Focus on self-assessment process and on controls to identify and correct problems

27. Contact Information Thomas P. Tierney, Director of Compliance Midwest Reliability Organization tp.tierney@midwestreliability.org (651) 855-1745

28. Questions?