Effects of Error, Variability, Testing and Safety Factors on Aircraft Safety

1. Effects of Error, Variability, Testing and Safety Factors on Aircraft Safety Erdem Acar, Amit Kale and Raphael T. Haftka eacar@ufl.edu akale@ufl.edu haftka@ufl.edu Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

2. Motivation • The FAA makes a distinction between error and variability through use of A-basis and B-basis properties. • A-Basis property is the value exceeded by 99% of population with 95% confidence. • Problems in acceptance of probabilistic design. We are interested to see whether the differentiating errors and variability may help. • We are interested to see how epistemic and alleatory uncertainty interact in determining the safety factor of aircraft. Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

3. Outline • Definition of uncertainties and safety measures considered • The error model • Simulation process for certification testing • Certification test effectiveness in terms of error, variability and average safety factor • Uncertainty in probability of failure • Concluding remarks Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

4. Error and Variability Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

5. Safety measures • FAA requirements • Safety factor (SF) = 1.5 • Certification tests: Testing the structural design for failure to compensate for ERROR (our interpretation!) • A-basis and B-basis material properties to account for VARIABILITY Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

6. Approach to the problem • Structural failure due to stress failure without damage propagation =P/Af ( is the point stress in any structural component) • A single test, which is a pass-fail certification test • Simulation of variability and error requires simulating the design of multiple aircraft and multiple models. • Monte Carlo simulation and analytical approximation used. Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

7. Error model • The deviation of actual load and stress values (fleet-average value) from the values calculated by the designer Error in load calculation (1) (2) Error in point stress analysis The designer uses Eq. (2) to calculate design thickness Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

8. Error in implementation • Deviation of average actual geometry and material properties from design specification Error in geometric parameters Error in material properties Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

9. Fleet-Average safety factor Fleet average of stress in a panel under correct design loads Fleet-average safety factor where cumulative error in safety factor for the average airplane (fleet-average) built by a company is safety margin for variability (brings 1.27 additional safety factor) Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

10. Error factor distributions Uniform distribution with zero mean Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

11. Variability • Variation from one aircraft to another in the fleet. For example, Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

12. Monte Carlo simulation N different aircraft models (Boeing 777, Airbus 320A) Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

13. Effect of certification on SF fleet The model is certified if • Mean initial : 1.907=1.5 x 1.27 • Mean updated : 1.932 • SAFETY IS IMPROVED ! • Since some unsafe designs fail in certification test. The use of A-basis properties gives and additional safety factor of 1.27. Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

14. Comparison of Monte Carlo and analytical approximation • Bayes Theorem is used to compute analytical approximation • Variability in geometric variables are approximated as lognormal • Certification testing does not affect error term ep Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

15. For low variability errors lead to safer design When the variability is very small! • SAFETY • IS IMPROVED ! Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

16. Effect of certification on Pf Introduce a new parameter k, • With variability, increase of k leads to increase of probability of failure • As error grows, Pf ratio becomes smaller indicating that the certification tests become more effective Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

17. Effect of variability Increase of variability leads to • Increase in probability of failure (A-basis not sufficient?!) • Increase in Pf ratio indicating that certification testing loses its efficiency Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

18. Effect of different safety measures (simpler error model) The usefulness of certification tests increases with - low safety factor - low variability - high error Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

19. Coefficient of Variation of Pf • Coefficient of variation of probability of failure is huge. • It may be difficult for an individual company to use the computed probability of failure. • However, for FAA it is O.K. since they are judged based on national average. Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

20. Small changes in SF may be sufficient for reliability based design • Deterministic and probabilistic design optimization of a simplified wing model. • For deterministic optimization, SF=1.5 and A-basis properties used. The use of both safety measure translates into an effective safety factor of 1.907. • The probabilistic optimization for fixed weight corresponding to deterministic optimum. • Aircraft companies may be given freedom to select conservative material properties to account for variability. Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu

21. Concluding Remarks • Safety is determined by error and variability. for error: SF and Cert. test --- for variability: A-basis Hence, certification tests are most effective for • low safety factors • high errors • low variability • Large coefficient of variation in probability of failure is found. • Safety factor may be useful for FAA to manage error. Aircraft companies may be given freedom to select conservative material properties to account for variability to improve safety. Structural and Multidisciplinary Optimization Group Dept. of Mechanical and Aerospace Engineering University of Florida eacar@ufl.edu