1 / 58

FAA Human Error Analysis Tool HESRA Workshop

FAA Human Error Analysis Tool HESRA Workshop. Schedule. 1300-1350 Introduction Project Background Human Error Analysis HESRA Overview 1400-1450 HESRA Method Detail 1500-1600 Documentation Discussion. Company. FAA User research and requirements gathering

ibtesam-javed
Download Presentation

FAA Human Error Analysis Tool HESRA Workshop

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. FAA Human Error Analysis ToolHESRA Workshop

  2. Schedule • 1300-1350 • Introduction • Project Background • Human Error Analysis • HESRA Overview • 1400-1450 • HESRA Method Detail • 1500-1600 • Documentation • Discussion

  3. Company FAA • User research and requirements gathering • User interface design hardware design • Human Error and Safety Risk Analysis (HESRA) • Human-computer interaction • Human factors research • Usability test and evaluation • Web site design and evaluation • Industrial and graphic design

  4. Domains In What Domains have we worked? • Command and control • Human-system integration • Medical devices • Patient safety • Mobile telecommunications • Commercial power • Consumer products • Aviation maintenance

  5. Project Background Background • Human error is implicated in 80-90% of all accidents and incidents across all domains • Incident investigations are after-the-fact • FAA looking for way to “predict” the likelihood of human error in design • HESRA offers a methodology to achieve this • Focus on ATC maintenance facilities • Evaluation of HESRA by applying to VSCS

  6. Human Error Analysis Human Error Analysis • Humans commit errors! • Error-free performance is not feasible • Factors that affect human error rates • Examine tasks • Identify high risk elements • Put resources where they are most needed

  7. Human Error Analysis Human Error Slip I intend to do the correct thing, but I do not do it properly. Mistake I decide to do something that is not correct. Violation (not an error) I intentionally violate a procedure, rule, or practice.

  8. Human Error Analysis Human Error Myths People who commit errors are lazy, stupid, inattentive, etc. • Fundamental Attribution Error • Most errors with really serious consequences are committed by highly-trained, -skilled, and -motivated people. Training can eliminate errors. • Errors are unintentional • Training affects intentional behavior • Training has no effect on most error-inducing factors

  9. Performance Shaping Factors Error Rates Increased By: • Time pressure • Environmental Factors • Heat/Cold • Noise • Confined space • Bad weather • Poor lighting • Poor equipment design • Poor user interface (UI) design • Confusing/ambiguous procedures • Fatigue • Illness • Personal/family issues • Management/labor conflicts

  10. HESRA Overview The basis for HESRA • Human Error and Safety Risk Analysis • Proactive human error analysis • Based on engineering model (FMEA) • Looking at human errors rather than component failures • Based on tasks rather than component functions • Three scales for each potential error • Likelihood of occurrence • Severity of outcome • Timeliness of recognition and recovery • Scales use nominal anchors • Goal is to produce ordered list of errors/outcomes

  11. HESRA What does HESRA do? • Identifies the relative likelihood of particular errors • Does not depend on past history, but can use this information • Relies on relative, ordinal scaling • Rank orders error modes • Identifies critical single component failures • Can utilize recovery factors - or not (same as FMEA) • Produces a task breakdown as a byproduct

  12. HESRA How does HESRA do it? • Starts with procedural and task breakdown • Relies on analysts to identify possible errors • For each error-cause pair, analysts assign ratings for • Likelihood of occurrence • Severity of outcome • Timeliness of recognition and recovery • Rating scales are 1-5, with 1=bad, 5=good • Ratings are multiplied to yield • Hazard Index (HI) = Likelihood X Severity • Risk Priority Number (RPN) = Likelihood X Severity X Detection • Error-cause pairs are sorted by HI, RPN, or both

  13. HESRA HESRA and the FAA Safety Management System (SMS) • SMS reverses scale meaning (as in MIL-STD-1629a) • Actually addresses human error risk assessment • For each error-cause pair, SMS assesses only • Likelihood of occurrence • Severity of outcome • Likelihood, severity, and recovery are de-coupled in HESRA • What is an error? FAA has very narrow definition.

  14. HESRA and the FAA What are the objectives of adapting HESRA to the FAA? • Provide FAA human factors staff with a tool that will allow them to evaluate system design and proactively predict elements of design that negatively influence human performance and safety. • Allow FAA to field better and safer systems that will protect passenger and crew safety, and improve the ability of maintainers to successfully perform the job

  15. HESRA and the FAA End of HESRA Overview Questions or Discussion?

  16. HESRA PROCESS

  17. HESRA Process Step 1 - Establish the Analysis Team Human Factors Specialist/Team Leader Maintenance/Operations Subject Matter Expert (SME) Trainer System Technical Specialist ATO Scientist ATO Manager (Optional)

  18. HESRA Process Human Factors Specialist • Understands the definition and foundations of human errors • Understands human perception, performance, and cognition • Understands effects of performance shaping factors • Understands compatibility of tasks with human capabilities and limitations • Able to interpret research and other empirical data • Needs good facilitation skills

  19. HESRA Process Maintenance/Operations SME • Working knowledge of the system/facility being analyzed • Understands the specific procedures to be analyzed • Has deep experience performing the actual procedures being analyzed • Understands scheduling and reporting • Has a realistic view of job tasks and errors

  20. HESRA Process Trainer • Has extensive experience teaching the types of tasks to be analyzed • Has extensive experience training maintainers on the system under analysis • For new systems, has broad and deep understanding of the system under analysis • Maintains communication links to field maintainers • Has broad knowledge of field incidents - both reported and unreported

  21. HESRA Process System Technical Specialist • In-depth knowledge of system functions, operations, and interactions • Can also fill rolls of maintainer and trainer • Can be a representative of the system developer • Particularly helpful for systems under development and not yet deployed

  22. HESRA Process ATO Scientist (optional) • Long association with operational systems • Broad view of how systems (and errors) can interact • Can also fill role of HF Specialist

  23. HESRA Process ATO Manager (optional) • Broad experience with various workers on the system under analysis • Can identify and explain errors that result from conflict or lack of coordination among managers and technicians • Has knowledge of the effects of errors on the overall ATC system • Can explain the effects of errors on management functions

  24. HESRA Process Team Tasks • Initial meeting • Establish ground rules • Agree on level of effort • Become familiar with system to be analyzed

  25. HESRA Process Step 2 - Familiarize Team with System • Team lead knows the least • Overview description by SME(s) • Physical walk-around • Visit to ATC facilities • Interviews with operators and maintainers • Walk through representative procedures

  26. HESRA Process Step 3 - Prioritize Procedures to Analyze • Review each candidate procedure • What’s the worst that can happen? • Additional criteria for selection

  27. HESRA Process Step 4 - Set Analysis Perspective User population Usage environment Performance shaping factors Overall complexity

  28. HESRA Process Performance Shaping Factors Bad • Time pressure • Fatigue • Multi-tasking • Noise • Physical exertion • Poor communication • Confusing terminology • Complexity

  29. HESRA Process Performance Shaping Factors Good • Well-designed user interface • Good communication links • Good training • Well-written procedures • Lack of time pressure • Quiet workspace • Well-rested technicians • Good labor-management relations

  30. HESRA Process Step 5 - Define Tasks Tasks vs. Steps • Tasks tell what to do, but not how to do it. Example: Identify the active server • Steps are individual “how to” actions. Example: Select “server map” from “View” menu. In “server” section, locate server labeled “active”. Etc.

  31. HESRA Process Step 5 - Define Tasks • ATC tasks are typically procedure-oriented. • Procedures contain on ordered list of tasks and steps. • Need to identify tasks and give each a name • Level of task detail varies Examples: “Remove paper from printer.” “At the VCSU Ops Console, select the <Ctrl> key twice.” • Analyze task detail and then abstract to proper level • What if the procedures don’t exist?

  32. HESRA Process The “General” Task • Not related to individual tasks or steps • Applies to the procedure as a whole • Not starting the procedure • Not completing the procedure • Doing the procedure out of order

  33. HESRA Process Step 6 - Define Steps • Identify steps required to complete each task • Definition of step • Correct level of detail • Enter steps into HESRA spreadsheet

  34. HESRA Process Step 7 - Define Errors, Causes, and Effects • Goal is to identify an exhaustive list of errors • The same error can have different causes • Identical errors likely have identical effects • Pre-fill errors and causes • Review step by step

  35. HESRA Process Exhaustive List of Errors • Undefined errors will not be analyzed • Tendency to skip low-likelihood errors • Separate identification of errors from rest of analysis • Perspective is “skeptical and evaluative” • No step is “too easy” for errors • Always include certain errors, e.g. “Fail to do step”

  36. HESRA Process Pre-fill Errors and Causes • Human Factors Specialist • Tedious process • Each error can have many causes • Internal and external factors • Effects and recovery factors

  37. HESRA Process Detailed Analysis • Team reviews each step-error-cause • Proceed on a step-by-step basis • Discussion of effects • Some errors never previously considered • Errors can have both system and human effects • Recovery factors Training Obviousness Built-in alarms Inability to perform subsequent steps Etc. • Relate to other steps and components

  38. HESRA Process Relating Errors to Other Steps • Very important aspect of analysis • Dependent errors • Conditional errors • Magnified or diminished effects • Changes in recovery • Noted in comments • New row in analysis matrix

  39. HESRA Process Step 8 - Assign Rating for Error Likelihood

  40. HESRA Process Assign Rating for Error Likelihood • Review each error-cause combination • Use existing data, if available • Skills or capabilities beyond typical human range • Team discussion and consensus • Same error, different task or step • Internal consistency • Performance shaping factors

  41. HESRA Process Step 9 - Assign Rating for Severity RatingCategoryDescription 5 Negligible No injury or equipment damage No significant effect on - safety - function/service - schedule 4 Marginal Minor injury or slight equipment damage Work around Loss of redundancy for non-critical component No delays Increased risk of more serious effects Minimal decrease of safety margin

  42. HESRA Process Step 9 - Assign Rating for Severity RatingCategoryDescription 3 Significant Moderate injury or equipment damage Loss of redundancy for a critical component Minor delays Increased maintainer workload Decreased safety margin Increased risk should remaining equipment fail Potential increased stress on remaining functional equipment

  43. HESRA Process Step 9 - Assign Rating for Severity RatingCategoryDescription 2 Critical Serious injury or moderate temporary loss of equipment function Moderate delays Increased maintainer workload Increased safety risk for maintainers Potential loss of A/C separation Brief reduction in local safety margin

  44. HESRA Process Step 9 - Assign Rating for Severity RatingCategoryDescription 1 Catastrophic Serious injury, death, permanent loss of one, or more, equipment function Extended loss of function/service Major delays Major increase in maintainer workload Increased safety risk for maintainers Loss of positive A/T control Extended reduction of safety margin

  45. HESRA Process Assign Rating for Severity • Worst-case scenario • Team discussions and consensus • Conditional errors and sequences of errors • Internal consistency • Effects of performance shaping factors

  46. HESRA Process Step 10 - Assign Rating for Detection and Recovery

  47. HESRA Process Assign Rating for Detection and Recovery • Composite of detection and recovery - Find and “fix” the error - Block the effects of an error • Influence of severity - Severity decreases detection and mitigation capability • Effects of performance shaping factors

  48. HESRA Process Step 11 - Calculate Hazard Index (HI) and RPN • Hazard Index = Likelihood X Severity • Risk Priority Number = Likelihood X Severity X Recovery • Automatically calculated by HESRA spreadsheet

  49. HESRA Process Step 12 - Analyze Criticality • Two forms of Criticality - HI Criticality - RPN Criticality • Categorization of HI and RPN calculated indices • Breakpoints are arbitrary, but meaningful • Categories typically related to action requirements • Automatic categorization by HESRA spreadsheet

  50. HESRA Process Hazard Index (HI) Criticality

More Related