1 / 12

TALES FROM THE TRENCHES

TALES FROM THE TRENCHES. TASK OUTLINE. INSPECTIONS APPLIED TO A TEST DEVELOPMENT PROJECT. Inspections were conducted to support Operational Flight Program (OFP) Test Script Development for the Avionics Integration Support Facility (AISF) AISF is an OFP software development and test facility

Download Presentation

TALES FROM THE TRENCHES

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. TALES FROM THE TRENCHES • TASK OUTLINE

  2. INSPECTIONS APPLIED TO A TEST DEVELOPMENT PROJECT • Inspections were conducted to support Operational Flight Program (OFP) Test Script Development for the Avionics Integration Support Facility (AISF) • AISF is an OFP software development and test facility • Supports the development and maintenance of several of Avionics Operational Flight Programs (OFPs) • OFPs had been previously developed by various Suppliers • OFP Software test procedures were required to be automated and consistent with each other to reduce manpower levels • Existing procedures varied from highly automated to entirely manual • Test environments varied from Supplier to Supplier • Availability of Supplier test bench documentation was limited • Existing OFP test procedures were converted into “OFP test scripts” • “Test scripts” were written using a high level BASIC-like language

  3. TEST SCRIPT DEVELOPMENT • Script Development Risk Factors • Size of Procedure • Level of Pre-existing Automation • Level of Supplier Documentation • Resident Expertise • Level of Expertise of Script Developer • Script Development Methodology • Analysis of conversion task • Identification of “Representative Test Scripts” • Conversion of “Representative Test Scripts” • Walkthrough of “Representative Test Scripts” • Conversion of remaining test scripts • Integration utilizing the “Representative Test Scripts” • Debug and test of remaining test scripts

  4. CONVERSION WALKTHROUGH • Two Phases • Internal Review (Informal with I&T members and Systems) • Formal Review (With Systems, I&T and Software) • Topics Discussed in Internal Review • Representative samples of scripts using available syntax • Identify “unknowns” • Topics Discussed in Formal Review • Samples of scripts, which fully cover all box functions, converted to ATU • Conversion Approach • Manual or translator to be developed • Pro/Cons if multiple approaches can be pursued, including schedule impacts • Risk areas • Metrics • Schedules

  5. TEST CONVERSIONWALKTHROUGH PROCESS Identify “Representative” Test Cases Develop “Top-Down” Functional Flows for Test Cases Develop ATL for Functions/ Utilities Conduct Informal Review with I&T and Systems Representative Revise “Top-Down” Structure/ Utilities Revise ATL for Selected Test Cases Conduct “Formal” Review with Management, Software, Systems, and I&T Obtain Approval on Test Conversion Approach Fully Develop “Representative” Test Cases and Deliver to Software for Checkout Checkout on Bench Software/Script Revisions Continue to Develop Remainder of Test Cases Verification of OFP Qual Capability

  6. WALKTHROUGH CHECKLIST • Overview Of Existing Vendor Test • Mapping Of Existing Vendor Test to AISF Test • Overview Of AISF Test, Including Names Of FTDS and Test Cases • User Interface (E.G. TUI Screen For Bench Configuration) - Formal Walkthrough Only • Simple Block Diagram (CTC, ITC, Emulator, Test Equipment) - Formal Walkthrough Only • Box Input/Output Identified by AISF S/W Variable Name or HEX Overwrite Syntax, depending on the implementation • Calling Trees for Script Procedures and Utilities for each FTD • General types of Inputs and Outputs, as well as Functional Descriptions of Script Procedures and Utilities

  7. WALKTHROUGH CHECKLIST (Con’t) • Traceability Matrix, with selected “Representative” Test Cases Bolded, and Bolded “X” for the “Representative” Function • Scripted “Representative” Test Cases using AISF ATL • Total Lines Of Estimated Script (Include Comments In Count) • Conversion Plan, Identifying Manual Conversion or Translator. If using a Translator, Identify Translator Development Plan and Translator Processing Requirements

  8. TEST SCRIPT INTEGRATION • Bench Integration and Risk Reduction of Test Script Development Effort focused around the integration and debug of “Representative Test Cases” • Integration Risk Factors • Number of Bench Functions • Experience Level of the Test Engineer(s) • Bench Integration order • In House Expertise • Overall Project Integration Effort • 430 K SLOS of test scripts • 1 M SLOC of simulation and support software • Eight OFPs • Eight LRU Benches

  9. IMPACT OF WALKTHROUGHS ON TEST SCRIPT INTEGRATION SUCCESS • Walkthrough Success Factors • Peer Review vs. Expert Review • Preparation of reviewers • Action Items Tracked and Closed • Completeness of Review • Project Completion Impacts • Budget Impacts • Schedule Impacts • Rework • Work-Arounds • Completion Criteria

  10. IMPACT OF WALKTHROUGHS ON TEST SCRIPT INTEGRATION SUCCESS (Con’t) • Premise: If Walkthroughs had no impact, then the Risk Factors for Integration would correlate with the Integration Success (...assuming all things being equal) • Conclusion: Successful Walkthroughs reduced integration risk and Project Completion Impacts

  11. LESSONS LEARNED • All Engineers should be trained in Walkthroughs and Peer Review Process • QA participation is crucial • CM participation is crucial • “Non-software” should be handled consistent with Software Best Practices • New engineers out of school are less resistant to Process Change • Make sure management, team leads and backups have “bought into” process • Don’t slack up on the later activities • Do inspections on all aspects of the development process • Don’t cut corners, even if you think you know what you are doing • Handle all elements of the system as if you are reviewing software • Reduces interface risk

  12. BOTTOM LINE • SEI Processes reduce risk when applied to • Systems Level Integration and Test Activities

More Related