Quantification Using the SPAR-H HRA Method: A Simple Exercise

1. Quantification Using the SPAR-H HRA Method:A Simple Exercise Ronald Laurids Boring, PhDHuman Factors Scientist, Idaho National LaboratoryVisiting Scientist, Halden Reactor Project Vinh Dang, PhD Head, Risk and Reliability GroupPaul Scherrer Institute

2. Three Mile Island Londonberry Township, Pennsylvania4:00am, Wednesday, March 28, 1979

3. Londonberry Township, Pennsylvania4:00am, Wednesday, March 28, 1979

4. Londonberry Township, Pennsylvania4:00am, Wednesday, March 28, 1979 The accident to unit 2 happened when the reactor was operating at 97% power. It involved a relatively minor malfunction in the secondary cooling circuit which caused the temperature in the primary coolant to rise. This in turn caused the reactor to shut down automatically. Shut down took about one second. At this point a relief valve failed to close, but instrumentation did not reveal thefact, and so much of the primary coolant drained away that the residual decay heat in the reactor core was not removed. The core suffered severe damage as a result. On March 30, it became necessary to vent a high pressure hydrogen bubble that was forming in the reactor core. This released approximately 12 millisieverts (mSv) of radioactive gas into the environment.

5. Londonberry Township, Pennsylvania4:00am, Wednesday, March 28, 1979 • Important Outcomes: • Increased awareness of the dangers of nuclear energy.Although 12 mSv of radioactive gas were released, extensive environmental monitoring as well as 20-year monitoring of 30,000 individuals living within 10 miles of plant revealed that the average exposure due to the plant was 0.08 mSv and the highest individual exposure was 1 mSv (equivalent dosage to a chest x-ray). • From a regulatory or industry standpoint, the plant worked as planned. Radiation was safely contained in the face of a core meltdown. • From the public standpoint, this was a disaster. No new plants planned for over 25 years.

6. Londonberry Township, Pennsylvania4:00am, Wednesday, March 28, 1979 • Important Outcomes: • Increased awareness of the importance of human factors. The operators were unable to diagnose or respond properly to the unplanned automatic shutdown of the reactor. Deficient control room instrumentation and inadequate emergency response training proved to be root causes of the accident. • As a result, the US Nuclear Regulatory Commission began a process of cataloguing all system interface and human performance issues that could increase plant risk. • Resulting disciplines are called probabilistic risk assessment(PRA) and human reliability analysis (HRA).

7. Two Levels of Precision • Conservative (screening) level useful for determining which human errors are the most significant detractors from overall system safety • An HEP for a modeled HFE may be set to 1 to determine if it is risk significant to the safety of the plant • Those found to be potentially significant contributors are analyzed in greater detail using more precise quantification

8. Quantification Concepts Nominal HEP • Default or general error rate, all things being equal Basic HEP • Nominal HEP adjusted for performance shaping factors Conditional HEP • Basic HEP adjusted for task dependency • Dependency = likelihood of error on one task increases the likelihood of error on a subsequent task • Mathematical correction to the basic HEP (usually increases the HEP because of previous error) Statistical Considerations • Every HEP has a measure of central tendency (a mean or median) coupled with the shape of the distribution and uncertainty (upper and lower bounds or standard deviations) • Distributions commonly in form of beta distribution (same as normal distribution, except on log-log axis)

9. Expert Estimation of HEPs A common technique for determining an HEP is to estimate its value by using a subject matter expert • What is the likelihood of failurefor this task? • Use pre-defined calibration points

10. Class Exercise • Estimate the HEP of the following tasks: • On your bus ride home, the bus hits a parked car • On your bus ride home, the bus driver is distracted • On your bus ride home, the bus driver speeds • On your bus ride home, someone fails to get off at the right stop

11. Issues with Expert Estimation • Subject matter experts may not be experts at producing probabilities • Generally, humans are not skilled at translating mental representations into quantities • Quality of information presented to the expert can greatly affect estimate • Experts often do not agree • In a group setting, one expert may dominate or influence others • In a group setting, it may be difficult to reach consensus • Experts may not be calibrated—even if they actually agree, they may not produce the same result

12. Improving Quantification Try to avoid exaggerated illusion of precision • An expert-generated HEP should not be a substitute for empirically derived data When no empirical data are available, use an HRA quantification method • Methods like SPAR-H are based on empirical data but are generalized enough to be flexible for most applications

13. SPAR-H Basics • SPAR-H = Standardized Plant Analysis and Risk Human Reliability Analysis Method (NUREG/CR-6883, 2005) • Uses worksheets or checklists to define factors that influence performance • Primarily used as a screening tool in HRA • Based on a cognitive/information processing model of human behavior • Quantification tool • Used to estimate HEPs • Quantification based on simplification and generalization of THERP method (NUREG/CR-1278, 1983) • THERP largely based on matching a pre-analyzed scenario from a nuclear power plant to the scenario being analyzed • SPAR-H generalizes these THERP scenarios according to performance shaping factors (PSFs)

14. SPAR-H PSFs • SPAR-H Worksheets are used to quantify HEPs by considering PSFs that may increase/decrease likelihood of error • Available time - Stress/stressors • Complexity - Experience/training • Procedures - Ergonomics/HMI • Fitness for duty - Work processes

15. SPAR-H Influences on HEPs • While performance is a complex process, SPAR-H utilizes these influences to estimate HEPs

16. SPAR-H Worksheet Types • The current SPAR-H method has worksheets for: • Diagnosis-type activities (i.e., cognitively engaging tasks) • Action-type activities (i.e., proceduralized, routinized, or action-oriented tasks) • Different modes of power operation are included • Full power operations (i.e., fault intolerant situation) • Low power and shutdown operations (i.e., fault tolerant situation)

17. SPAR-H Nominal HEPs • Assumption that for practiced, proceduralized activities, humans have a default error probability • This is the nominal HEP • Diagnosis: NHEP = 1E-2 (0.01 or 1 in 100 occurrences will result in an error) • Action: NHEP = 1E-3 (0.001 or 1 in 1000 occurrences will result in an error) • Only use estimated HEPs in the absence of actual performance data!

18. Nominal HEPs and PSFs • PSFs serve as multipliers to increase or decrease the HEP compared to the nominal rate • e.g., available time PSF for diagnosis and action

19. Barely Adequate Time Example • If there is barely adequate time (2/3 required time), the NHEP is multiplied by 10 • Diagnosis = 1E-2 x 10 = 1E-1 (1/10 chance of failure) • Action = 1E-3 x 10 = 1E-2 (1/100 chance of failure)

20. Extra Time Available Example • If there is extra time (up to 2 times the required time), the NHEP is multiplied by 0.1 • Diagnosis = 1E-2 x 0.1 = 1E-3 (1/1000 chance of failure) • Action = 1E-3 x 0.1 = 1E-4 (1/10000 chance of failure)

21. Available Time Notes • If there is no information available about the PSF, a nominal or insufficient multiplier is assumed • If there is inadequate time, HEP = 1.0 (no multiplier)

22. Adjustment Factors in SPAR-H • Adjustments are made to the HEP for: • Number of PSF modifiers • for three or more negative influencing PSFs, adjustment applied to decrease overall HEP • Dependency • if an earlier event primes (increases the likelihood) of a subsequent event, the individual event HEPs are increased to account for the level of dependency

23. Class Exercise • Example of a medical error in radiation treatment of a patient taken from Set Phasers on Stunby Steven Casey: • Ray Cox, 33, receiving ninth radiation therapy treatment after removal of cancerous tumor from his shoulder. Everything starting to become fairly routine, and he was quite comfortable with Mary Beth, his radiotherapy technician, and the THERAC-25 radiotheraphy machine. Ray lied face down on table. Mary Beth positioned the THERAC-25 and went into the control room. Mary Beth used a computer terminal to enter commands on THERAC-25. The video and audio between the patient room and the control room were not working. There were two modes: a high-power x-ray dose to radiate tumors and a low-power electron beam for subsequent treatment. Mary Beth accidentally put it in x-ray mode by typing [X] but then corrected it to electron mode by moving the cursor up and typing [E]. She then pressed [RETURN] to administer the treatment.

24. Class Exercise (Continued) • Set Phasers on Stun(Continued): • No one had every changed an [X] to an [E] before in this manner. Atomic Energy Canada, who developed the THERAC-25, had not anticipated this way of changing the mode. This error not only switched the THERAC-25 into x-ray mode, it disabled a metal plate that limited the intensity of the x-ray. Ray Cox’s intended dose of 200 rads actually became 25,000 rads! Mary Beth activated the first beam but received an error message that sounded like the beam had not been applied. She tried again two more times. The first time, Ray Cox heard a frying sound and felt an excruciating stabbing pain in his shoulder. Rolling in pain, the THERAC-25 fired again, this time into his neck. Screaming in pain, a third dose went through his neck and shoulder. He ran out of the treatment room. Mary Beth, meanwhile, was unaware what had happened, but the THERAC-25 reported Ray had only received 20 rads. In fact, he had received 75,000 rads. Four months later, Ray died due to radiation overdose. He remarked, “They forgot to set the phaser on stun!”

25. Class Exercise (Continued) • Set Phasers on Stun(Continued): • What Was Supposed to Happen • Set patient on table • Position THERAC-25 • Go to control room • Enter prescribed dose • Activate dose • Retrieve patient • What Actually Happened • Set patient on table • Position THERAC-25 • Go to control room • Enter prescribed dose • Correct wrong entry • Activate dose • Error message • Go back and reactivate • Error message • Go back and reactivate • Patient flees

26. Class Exercise (Continued) • Set Phasers on Stun(Continued): • What Was Supposed to Happen • Set patient on table • Position THERAC-25 • Go to control room • Enter prescribed dose • Activate dose • Retrieve patient • What Actually Happened • Set patient on table • Position THERAC-25 • Go to control room • Enter prescribed dose • Correct wrong entry • Activate dose • Error message • Go back and reactivate • Error message • Go back and reactivate • Patient flees What is the likelihood for entering and giving the wrong dose?

27. Class Exercise (Continued) What is the likelihood for entering and giving the wrong dose? • First, consider the relevant PSFs from SPAR-H • Available time - Stress/stressors • Complexity - Experience/training • Procedures - Ergonomics/HMI • Fitness for duty - Work processes • Determine which PSFs apply, and which do not

28. Class Exercise (Continued) What is the likelihood for entering and giving the wrong dose? • Next, consider if it is a diagnosis (cognitive)or action (behavior) • Finally, consider the levels of applicable PSFs • Use the numbers in parentheses on this table • Calculate the Basic HEP • Nominal HEP (1E-2 or 1E-3) xTime x Stress x Complexity xExperience x Procedures xErgonomics x Fitness for Duty xWork Processes • Correct for too many PSFs • Adjust for Dependency

29. Class Exercise (Continued) You have completed your first HRA! • Typically, it’s not quite that simple • SPAR-H is one of the simpler quantification tools • Since it was designed as a screening tool, the numbers are always treated as approximations • Other methods may be more applicable to other applications • Since SPAR-H was designed for nuclear power plant operations, the method may not generalize to other applications • It’s easier to do a bad analysis in SPAR-H than with other methods • Important to document all analysis assumptions • Still, a powerful tool that gives you a “ballpark” estimate of risk involving human activities • This can be used to identify and prioritize potential errors before they become actual errors!