Testing safety-critical software systems

1. Testing safety-critical software systems Marcos Mainar Lalmolda Quality Assurance and Testing 20th November 2009

2. Contents • What a safety-critical software system is • Standards • Programming features and languages • Approaches on design • Testing • Conclusion

3. What a safety-critical software system is • A safety-critical software system is a computer system whose failure or malfunction may severely harm people's lives, environment or equipment. • Some fields and examples: • Medicine (patient monitors) • Nuclear engineering (nuclear power station control) • Transport (railway systems, cars anti-lock brakes) • Aviation (control systems: fly-by-wire) • Aerospace (NASA space shuttle) • Civil engineering (calculate structures) • Military devices • Etc.

4. Safety-criticalStandards • Industries specific • Medical device software: IEC 62304 • Nuclear power stations: IEC 60880 • Aerospace: AS9100A • Airbone: DO178B • … • Scale of 5 safety integrity levels: 4 is very high, 0 not safety related. • Safety engineering

5. Programming features and languages (I) • General principle: Try to keep the system as simple as possible. • Programming features not recommended: • Pointers and dynamic memory allocation/deallocation. • Unstructured programming (gotos) • Variant data • Implicit declaration and initialisation • Recursion • Concurrency and interrupts

6. Programming features and languages (II) • Features which increase reliability: • Strong typing • Run time constraint checking • Parameter checking • Language to be avoided: C • Language recommended: Ada • Ada subset for safety-critical software: SPARK • Other languages: increased overhead

7. Approaches on design • Formal methods • Assume that errors exist and design prevention and recovery mechanisms. • “Program verification does not mean error-proof programs […]. Mathematical proofs can also be faulty. So whereas verification might reduce the program-testing load, it cannot eliminate it” (F.P. Brooks, No Silver Bullet, 1987).

8. Testing safety-critical software systems (I) • Basic idea: Identify hazards as early as possible in the development life-cycle and try to reduce them as much as possible to an acceptable level. • Remember: Always test software against specifications! • Independent verification required • If formal methods have been used then formal mathematical proof is a verification activity. • Already known techniques used for typical systems • White box testing • Black box testing • Reviews • Static analysis • Dynamic analysis and coverage

9. Testing safety-critical software systems (II) • Specific procedures and techniques from safety engineering: • Probabilistic risk assessment (PRA) • Failure modes and effects analysis (FMEA) • Fault trees analysis (FTA) • Failure mode, effects and criticality analysis (FMECA) • Hazard and operatibility analysis (HAZOP) • Hazard and risk analysis • Cause and effect diagrams (aka fishbone diagrams or Ishikawa diagrams)

10. Probability Risk Assessment Hazard Severity Probability Risk Risk Criteria Risk Reduction Measures Tolerable? No Yes *From Safety-Critical Computer Systems – Open Questions and Approaches presentation, Andreas Gerstinger, February 16, 2007, Institute of Computer Technology, Wien 10

11. Fault tree analysis (FTA) • A graphical technique that provides a systematic description of the combinations of possible occurrences in a system which can result in an undesirable outcome (failure). • An undesired effect is taken as the root of a tree of logic • Each situation that could cause that effect is added to the tree as a series of logic expressions. • Events are labelled with actual numbers about failure probabilities. • The probability of the top-level event can be determined using mathematical techniques.

12. An example of a Fault tree *From http://syque.com/quality_tools/toolbook/FTA/how.htm

13. Conclusions • Complex subject • Suitably trained and experienced people are key to the success of any software development. • Main objective of testing techniques: minimise risk of implementation errors. • Above all, the best way to minimise risk both to safety, reliablity and to the timescale of a software project is to keep is simple.

14. Questions ¿?

15. References • Wikipedia. http://en.wikipedia.org • IPL Information Processing Ltd, An Introduction to Safety Critical Systems, Testing Papers. http://www.ipl.com/include/download/CookieRequestPage.php?FileID=p0820 • IPL Information Processing Ltd, An Introduction to Software Testing, Testing Papers. http://www.ipl.com/include/download/CookieRequestPage.php?FileID=p0826 • Evangelos Nikolaropoulos, Testing safety-critical software, Hewlett-Packard Journal, June 1997. http://findarticles.com/p/articles/mi_m0HPJ/is_n3_v48/ai_19540814/?tag=content;col1 • Frederick P. Brooks, Jr. , No Silver Bullet: Essence and Accidents of Software Engineering, 1986. • Andreas Gerstinger, Safety-Critical Computer Systems – Open Questions and Approaches presentation, February 16, 2007, Institute of Computer Technology, Wien. • Fault Tree Analysis: How to understand it. http://syque.com/quality_tools/toolbook/FTA/how.htm