SOUND METHODS and EFFECTIVE TOOLS for ENGINEERING MODELING and ANALYSIS _________________. by David Coppit, College of William and Mary, and Kevin J. Sullivan, University of Virginia Proceedings of the 25th International Conference on Software Engineering Portland, Oregon - May 3-10, 2003.
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
by David Coppit, College of William and Mary,
and Kevin J. Sullivan, University of Virginia
Proceedings of the 25th International Conference on Software Engineering
Portland, Oregon - May 3-10, 2003
Presentation by Bryan E. Bloss - University of Central Florida, Nov. 2003
Designing Software Tools for Formal Design
Engineering modeling & analysis methods
are based on modeling languages for
describing systems, with semantics for
mapping expressions (models) to
estimates of system properties (results).
To be safe and effective, a modeling method requires
a language with a validated semantics; feature-rich,
easy-to-use, dependable tools; and low engineering costs.
Hampered by shortcomings in software engineering & languages, today we lack adequate means to develop such methods.
- Finding a cost-effective way to ensure
semantic soundness of a complex method
- Using Package-Oriented Programming (POP) to
produce easy-to-use, functionally-rich tools from
available software packages (such as MS Office)
Results: A package-based tool “Galileo” is evaluated
favorably by NASA engineers, and development of
“Nova”, a similar tool based on a formal semantics,
proves the cost-effectiveness of a combined approach
Why are they important?
Specification is more fundamental than implementation.
Without a formal specification:
- Validation is difficult
- No basis for a definitive user reference document
- Programmers are left to make uninformed semantic
decisions; unable to thoroughly test correct functioning.
Tools used in the design of safety critical systems should be treated as critical engineering components.
Our inability to develop low-cost, easy-to-use tools can thus be seen as a positive safety mechanism, but far from ideal.
Safety Example: 1996 alert from U.S. Nuclear Regulatory Commission warned of significant errors in several tools which had been adopted for use in nuclear reactor design & analysis
Another Example (not in paper): Crater analysis tool, used inappropriately during flight STS-107 to analyze foam damage
Developing the Galileo Tool for DFT Analysis
Observation: Most applications devote less than 10% of their code to the core function of the system!
90% is devoted to superstructure-- support functions such as text & graphical editing, data validation, etc.
Package-Oriented Programming (POP) is intended to save time in creating superstructure; frees more resources for the critical design activity: applying formal methods to define & validate the syntax and semantics of the modeling language
The Application: Dynamic Fault Trees (DFT)
Graphcal representation of every
conceivable sequence of events
that could cause a system to fail.
Each leaf is a basic event; internal
gates define relationships leading
to system failures at upper levels.
Static trees model how event
combinations lead to failures;
Dynamic trees are order-sensitive.
The Problems With Current DFT Languages
During development of Galileo tool, a non-trivial error was found in the underlying DFT language, DIFtree, where probability of a masked (hidden) failure wasn’t correctly computed
Also, DIFtree’s informal specification had left ambiguities on how to handle special cases; prior software implementations answered these questions inconsistently in different parts of the program.
And formal validation was time-consuming, due to lack of automation in available syntax & theorem-prover tools, and slow run-time perfomance
Worse, the theorem-prover tool required too much user expertise; guidance often needed from the tool’s author
Like Galileo, uses POP components from MS Office for fault tree editing:
- Word for text editing
- Visio graphical editor (enhanced for DFT modeling constructs)
- Excel for computational results
Will allow even more emphasis on formalization & validation than in Galileo
New version of Galileo tool funded by NASA Langley Research Center, to support new modeling & analysis constructs and be usable in practice
Featured in three workshops:
1st- Managers & engineers from several NASA divisions
2nd- Space Station engineers only
3rd- Space Shuttle engineers only
A short survey (34 questions) and an in-depth survey (77 questions) were offered to engineers, to evaluate user perceptions of usability & features
Feedback indicated that usability was same or better than other tools!
Also confirmed that dependability is crucial; a formal specificaltion of the modeling language was second only to a comprehensive test suite as a means for increasing trust.
Strengths:A well-informed overview of the authors’ experience with developing a new software toolset for practical engineering, while emphasizing formal validation methods. Many implications for the design of other reliability-critical software applications.
Weaknesses:Little information on costs saved by POP method; little detail on formal proving methods. Also, the more advanced NOVA tool had not been user-tested at time of publication, so final results aren’t known.