1 / 59

NASDA IV&V ………..

NASDA IV&V ………. July 29th, 2003 @NASA IV&V Facility Naoki Ishihama, NASDA Hideki Nomoto, JAMSS/MIT Haruka Nakao, JAMSS. Contents. Introduction of NASDA IV&V approaches Project IV&V activities’ status and findings Satellites IV&V (SELENE, WINDS) JEM Pre-Shipping IV&V JEM IV&V HTV IV&V

courtney
Download Presentation

NASDA IV&V ………..

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. NASDA IV&V ……….. July 29th, 2003 @NASA IV&V Facility Naoki Ishihama, NASDA Hideki Nomoto, JAMSS/MIT Haruka Nakao, JAMSS

  2. Contents • Introduction of NASDA IV&V approaches • Project IV&V activities’ status and findings • Satellites IV&V (SELENE, WINDS) • JEM Pre-Shipping IV&V • JEM IV&V • HTV IV&V • Centrifuge IV&V • H-IIA Safety Range Ground system IV&V • New Methodology Research • Software QA activity and Process Improvement

  3. NASDA IV&V Functions IV&V Planning • Project IV&V (Assessment ReportProject Evaluation) • Development Independent Assessment • Shipping Validation, Pre-Launch Review • Non-conformance Investigation/Assessment • Research of IV&V methodology and tool • Criteria • Handbook • Tools • Software Process Improvement • Software Quality Assurance 1. Introduction of NASDA IV&V approaches

  4. IV&V approaches 1. Introduction of NASDA IV&V approaches

  5. Satellite IV&V Activity- WINDS, SELENE - SELENE WINDS 2. Project IV&V activities’ status and findings

  6. Satellite IV&V Project Characteristic • Basically non 2FT system (Fail safe, 1 fail operative) • One third of software/firmware are re-use. • High level specification defines detailed control theory and logic. • High speed calculation has become to be required. 2. Project IV&V activities’ status and findings

  7. Modeling of Satellite Attitude Control Software Requirement using UPPAAL • Real-time Application • Hard to deal with time constraints “What” and “How” should we do? • We used UPPAAL. • UPPAAL: • Model networks of extended timed automata with typical data • Verify an invariant and reachability of the model 2. Project IV&V activities’ status and findings

  8. Modeling of Satellite Attitude Control Software Requirement using UPPAAL Demo #1 ! 2. Project IV&V activities’ status and findings

  9. Pre-shipping Verification for Japan Experiment Module (JEM) Software 2. Project IV&V activities’ status and findings

  10. Project Characteristic All Development Activity had been done. Reverse Engineering Monstrous and complex on-board system relatively (DB) Need tool support Long term development and operation Need to keep the design rationale and records Pre-shipping Verification for JEM Software 2. Project IV&V activities’ status and findings

  11. Equivalency Checking Tool of source code (ECT) • Equivalency Checking Tool of source code (ECT) is a reverse engineering tool that generates a flow diagram from a source code. • Modifying GCC(GNU Compiler) 2.95.3 and DOT included in GRAPHVIZ tool set 2. Project IV&V activities’ status and findings

  12. Equivalency Checking Tool (ECT) Outline of tool procedure 2. Project IV&V activities’ status and findings

  13. Equivalency Checking Tool (ECT) void start_waiting_jobs () { struct child *job; if (waiting_jobs == 0) return; do { /* Check for recently deceased descendants. */ reap_children (0, 0); /* Take a job off the waiting list. */ job = waiting_jobs; waiting_jobs = job->next; /* Try to start that job. We break out of the loop as soon as start_waiting_job puts one back on the waiting list. */ } while (start_waiting_job (job) && waiting_jobs != 0); return; } Flow Diagram Reverse Engineering Source Code

  14. void start_waiting_jobs () { struct child *job; if (waiting_jobs == 0) return; do { /* Check for recently deceased descendants. */ reap_children (0, 0); /* Take a job off the waiting list. */ job = waiting_jobs; waiting_jobs = job->next; /* Try to start that job. We break out of the loop as soon as start_waiting_job puts one back on the waiting list. */ } while (start_waiting_job (job) && waiting_jobs != 0); return; } Flow Diagram Reverse Engineering Source Code Equivalency Checking Tool (ECT) Demo #2 !

  15. Equivalency Checking Tool (ECT) digraph start waiting jobs{ label=”job.c:1583: void start waiting jobs()”; node [shape=box]; edge [weight=100]; start [shape=ellipse, height=0.01, label=”START”]; subgraph if 1586{ node [shape=box]; edge [weight=100]; Na1586 [shape=diamond, label=”1586: if (waiting jobs==0)\n”... start -> Na1586; Nb1587 [shape=ellipse, label=”1587: return;”, fontsize=14]; Na1586 -> Nb1587 [taillabel=”Y”]; } ……. } Medium Code 2. Project IV&V activities’ status and findings

  16. Lesson and Learned • Flow diagram and additional information can be clearly compared with detailed design specification to determine the equivalency and differentials. • realize misunderstanding and miscommunication between programmer and system designer 2. Project IV&V activities’ status and findings

  17. Light weight code checking tool: Splint • Comment (warning) is different with compiler. • Most security attacks exploit instances • Many of these flaws could be detected and eliminated before software is deployed • Not been integrated into the software development We used the Splint. • Lightweight static analysis to detect common security vulnerabilities • Coding mistakes (to avoid specific cord pattern which caused the past non-conformance) 2. Project IV&V activities’ status and findings

  18. Benefit of Lint tool GCC compiler can’t find this area.

  19. ? …….. Light weight code checking tool: Splint void attitude_jobs () { …… …… switch (id_flg) { case 1: control(); …… case 2: stop_attitude(); …… case 3: restart_attitude(); …… …… } But case 4 ??? case 5 ??? ….. default ??? undefined Who am I? Where am I? What should I do? GCC: No warning!!! 2. Project IV&V activities’ status and findings

  20. ? …….. Light weight code checking tool: Splint void attitude_jobs () { …… …… switch (id_flg) { case 1: control(); …… case 2: stop_attitude(); …… case 3: restart_attitude(); …… …… } Demo #3 ! But case 4 ??? case 5 ??? ….. default ??? undefined Who am I? Where am I? What should I do? GCC: No warning!!! 2. Project IV&V activities’ status and findings

  21. Japan Experiment Module (JEM) IV&V activityOur first target of IV&V efforts:Wrap-up of these 5 years 2. Project IV&V activities’ status and findings

  22. The biggest issue of JEM Software • Complexity of Automated FDIR • Over 150 procedures which could conflict each other when executed simultaneously. if (MTL temp > max) stop pump-a close valve-a shutdown payload racks set pump-b 3500rpm open cross-over set pump-b 7000rpm if (MTL flow rate < min) stop pump-a close valve-a set pump-b 3500rpm open cross-over set pump-b 7000rpm if (NASA_H-EX_M is error) shutdown payload racks set pump-a 3500rpm open H-EX bypass valve open cross-over set pump-a 6000rpm 2. Project IV&V activities’ status and findings

  23. Background • Formerly simple procedures had evolved into awfully complicated programs as system/safety constraints were recognized by the designers. @ CDR if (MTL flow rate < min) mask LTL flow rate check mask MTL temperature check if (current mode != 1WCL) if (NASA H-EX-MTL != error) if (current flow rate > 3500) stop low priority procs wait 15 sec send message "MTL-stop procedure going on. Do not bother me." stop pump-a else ...... @ PDR if (MTL flow rate < min) stop pump-a close valve-a set pump-b 3500rpm open cross-over set pump-b 7000rpm 2. Project IV&V activities’ status and findings

  24. Initial JEM IV&V approach (~ 1999) • Build a dynamic simulator to check conflict • Difficulties of reproducing high fidelity execution environment (especially for fluid dynamics on zero G) • Cost of execution was enormous for good coverage of possible TIMING • Corrected JEM IV&V approach (2000~) • Build state-machine model for static analysis • Focus more on the software architecture • Focus more on the development of analysis algorithm • Provide top-down overhaul option (It gives cheap way in the end in many cases) 2. Project IV&V activities’ status and findings

  25. Examples (Thermal control FDIR redesign) • Complete matching survey for FDIR • procedures coupling • Simply extract critical commands and their execution conditions to build a model • Check conflicts by simple matching algorithm for every possible coupling of the commands • Detected over 800 conflicts for the first round • Eliminated in the second round Iteration Cycle • Overhaul of the architecture performed for : • Parallel execution control mechanism • Scratch building the sequence and grouping 2. Project IV&V activities’ status and findings

  26. Examples (File Transfer Protocol) • Back to basics • Simply build conventional state transition matrix from the design documents • Apply normalization techniques for the table which is quite popular in the relational database design • Check incomplete combinations of conditions • Detected over 20 incomplete conditions • Eliminated in the second round Solution was given by simply populating the table rather than spending long time with whacking a mole of test non-conformances. 2. Project IV&V activities’ status and findings

  27. Lessons we learned • Think static • Static analysis eliminates the difficulty of timing issue • Static model reveals architectural solutions more explicitly than dynamic analysis • Static model inspires us to be creative and fun • Think basic • For the post-CDR systems, everyone's focus is distributed to different details and could fall into hell of complexity. Keep ourselves harmonized by looking back to the architecture. • Don't rely on the super fancy-sexy analysis from the preliminary risk assessment stage. They are expensive. 2. Project IV&V activities’ status and findings

  28. H-ⅡA transfer vehicle (HTV)IV&V activity 2. Project IV&V activities’ status and findings

  29. Finding • FDIR methodology validation using formal model revealed significant numbers of undefined failure modes (especially in quintuple voting mechanism among dissimilar navigation sensors). • Root cause of the design flaw: • FDIR design originally started from bottom-up approach for each combination of failures • The numbers of combinations inflated as the software was adapted to additional devices and more safety/mission constraints 2. Project IV&V activities’ status and findings

  30. Issue • Quintuple voting mechanism shall handle 210 of • combinations of failures Unrealistic to have huge table to search (resource constraints) → ★Top-down study for a dedicated law to realize a feasible solution was required. 2. Project IV&V activities’ status and findings

  31. × SensorA SensorE SensorB Agree SensorC SensorD Disagree Solution • “Graph algorithm” to detect Failed Sensor • Rule: “Identify failure modes by identifying closed triangle(s) in the chart” • 5 categories of failures • Single failure • Two failures • More than three failures • Unidentifiable failure • Incredible failure 2. Project IV&V activities’ status and findings

  32. Merit • Using “Graph algorithm” • Simple and fast(Grouping of patterns , no large table search needed) • Mathematically complete • Reusable for any kinds of similar voting logic 2. Project IV&V activities’ status and findings

  33. LSG CR Centrifuge Rotor (CR) and Life Science Grove-box (LSG) IV&V activity 2. Project IV&V activities’ status and findings

  34. System Overview • Centrifuge is the experiment module for the life-science • Centrifuge Rotor (CR) : CR is rotational equipment for gravitational experiment The target S/W : Rotor Control Unit (RCU), (C-RIC), Laptop Hazardous Condition related with software is • Loss of spin control Catastrophic • Contact of rotating equipment Catastrophic • Rotor Dynamics hazard Catastrophic • Life-Science Grove-box (LSG): LSG provides an enclosed environment for biological specimens and chemicals Hazardous Condition related with software is • Specimen death Critical • Lost of experiment data Critical 2. Project IV&V activities’ status and findings

  35. Software Requirement Specification System Specification (software part) IV&V Process overview Development Process Natural Language Document Analysis Formal Model ・Inconsistency ・Completeness ・Reachability PRR 1st Step 2nd Step ・Inconsistency ・Completeness ・Traceability SRR 2. Project IV&V activities’ status and findings

  36. The 1st step- Modeling Natural language definition • AA Mode • The CR shall (REQ-01001) transfer mode from every mode without Initialization Mode or B Mode to AA Mode by command. Concerns: The ambiguous documentation and inconsistency often cause the former accident and incident. Natural language based Model AA Mode (REQ-01001) ‘CR is Stanby_Mode’ and ‘AA_Mode_Command is ON’ Or ‘CR is AA_Mode’ and ‘AA_Mode_Command is ON’ Or ‘CR is AB_Mode’ and ‘AA_Mode_Command is ON’ => CR is AA_Mode Improvements: We can analyze it formally with automated tools (Static Analysis) Formal Model (Matrix) CR = AA_Mode

  37. The 2nd step - Analysis • ★Consistency Analysis on documentation ★Completeness Analysis on documentation →DEMO #4 • ★Traceability Analysis →DEMO #5 • ★Hazard Reachability Analysis →DEMO #6 2. Project IV&V activities’ status and findings

  38. Natural Language Modeling and Analysis Tools (Demo #4) Tools: (run with gtk on multiplatform) Function • Natural Language Modeling • Model Checking • Consistency and Completeness Analysis Component Windows • Editor Window • Model Viewer • Analysis Result Viewer 2. Project IV&V activities’ status and findings

  39. Editor Window (modeling window) chapter section Rule word State AA Mode (REQ-01001) ‘CR is Stanby_Mode’ And ‘AA_Mode_Command is ON’ Or ‘CR is AA_Mode’ and ‘AA_Mode_Command is ON’ Or ‘CR is AB_Mode’ and ‘AA_Mode_Command is ON’ => CR is AA_Mode 2. Project IV&V activities’ status and findings

  40. Modeling Viewer Analysis Result Viewer 2. Project IV&V activities’ status and findings

  41. About Consistency Analysis Condition B Condition A Condition C Last State Next State • Analysis which check that there is not two or more states transitions (Ex: Next State include the condition B and condition C….) when system condition transit from last state to next state. 2. Project IV&V activities’ status and findings

  42. About completeness analysis • Analysis which check that the state corresponding to the defined conditions surely exists • The state of a system will be set to Unknown Mode when the defined conditions do not belong to all the state. Mode Parameter T:True F:False 2. Project IV&V activities’ status and findings

  43. For example of completeness analysis: Mode α Yes = “T” No = “F” β β Yes Yes No No γ γ γ γ Yes No No No Yes No Yes Yes E E D ? C C A B Mode αβγ TTT TTF TFT TFF FTT FTF FFT FFF Unknown Mode 2. Project IV&V activities’ status and findings

  44. Traceability of functions described in the Document at different level (e.g. requirements and specifications) must be confirmed → Evaluate automatically by TRAD tool PREPARATION: Translate functions to Formal model “Natural-Language Model” Example [CSM-01514] CR_Mode in Mode Rotation_Mode Operational_Mode_Transfer_CMD is ON (from Laptop) -> CR_Mode in Mode Operational_Mode DEMO #5TRAD(Traceability Diagnosis) Tool 2. Project IV&V activities’ status and findings

  45. EXECUTION: (1) Conversion Formal Model of each document to UML Sequence Diagram in every function (2) Evaluate similarity of two Diagrams → Traceabirity Example RESULT: - Evaluate Traceability accurately in a very short time - Found miss-traced functions based on Traceability System Laptop CMD is ON Mode Change DEMO #5 TRAD(Traceability Diagnosis) Tool 2. Project IV&V activities’ status and findings

  46. Preparation: <1> Formal Modeling of software specification <2> Formal Modeling of the hazard Objective of the integration: ★Generate software fault tree from system hazard ★Search for single failure point ★Search for state variables which eliminate hazard ★List up initial conditions to lead to the hazards DEMO #6Reachability Analysis Environment (RAE) 2. Project IV&V activities’ status and findings

  47. H-ⅡA Flight Safety Control System (SIRIUS) IV&V activity 2. Project IV&V activities’ status and findings

  48. System Overview • “SIRIUS” is the ground range safety control system for H-IIA rocket • SIRIUS provides • -Flight Path • -Area affected by explosion • -Attitude Inputs from -Radar -Telemeter -etc… Outputs for ground safety decision SIRIUS requirements ★Hard real-time (msec Synchronized) ★Non-Stop (of dozens of computers) ★ Built on top of commercial OS 2. Project IV&V activities’ status and findings

  49. Analysis Approach • Hard Real Time on top of commercial OS • Extensive analysis of code implementation for: • Timing control parameters (Detailed validation of unit test results) • Signal handling mechanism (Graphical sequence modeling) • Global variable handling (Automated tool to exhaustive search for potential conflicts) • Frequently updated for specific requirements of each launch • Line by line comparison for all code from the previous version at the launch site 2. Project IV&V activities’ status and findings

  50. Findings • Findings • Design flaw of process synchronization • Fluctuation of CPU load due to commercial OS system service • Lack of exception handling • Task priority inversion • Global variable conflicts • System configuration management problem 2. Project IV&V activities’ status and findings

More Related