1 / 28

Testing Workflow Purpose

Enhance software development by implementing a thorough testing workflow. Plan, design, and automate tests across iterative phases for optimal results. Monitor outcomes, refine models, and evaluate defects for continuous improvement.

jbryan
Download Presentation

Testing Workflow Purpose

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Testing Workflow Purpose • Purpose • Verify the result from implementation by testing each build • Plan the tests in each iteration • Integration tests for every build within theiteration • System tests for the end of the iteration • · Design and implement tests by creating • test cases that specify what to test • test procedures that specify how to perform the tests • executable test components to automate the tests

  2. Testing Workflow Flow

  3. Testing Workflow with Phases

  4. Testing Workflow in Phases Generally, when there is an implementation result, there is a test --- so Test at each build Inception phase: initial test planning, prototype testing Elaboration phase: test architectural baseline Construction phase: significant testing at each build Transition phase: re-test fixes and regression tests

  5. Testing Workflow Other Tasks Regression tests: in a new build, re-apply tests from old builds to make sure nothing “broke” in new build Evolve test model create new test cases for next build refine prior test cases into regression tests remove obsolete tests and corresponding test procedures and components

  6. Testing Workflow Models Test model artifacts Test cases Test procedures Test components Test subsystem packages for complex test Other artifacts Test Plan Defects Test Evaluation

  7. Testing Workflow Model TEST MODEL Describes how executable components (e.g., builds) in the implementation model are tested by integration tests and system tests Describes how specific system aspects are to be tested e.g., usability testing of user interface or user manual

  8. Testing Workflow – Test Case TEST CASE Specifies one way to test the system What to test (requirements) With which inputs Expected results Test conditions and constraints

  9. Testing Workflow Test Cases TEST CASE EXAMPLES Test a use case or a scenario in a use case (black box)§ actor interaction, pre-conditions, action sequences, post-conditions Test a use-case realization (white box) verify interaction between components. Installation tests, configuration tests, negative (abuse) tests, stress tests Organize similar tests in input/result matrix Family of test for different paths through a use case

  10. Testing Workflow Special Tests Installation tests verify the system can be installed on the customer platform and that it executes correctly when installed Configuration tests verify the system works correctly in different configurations Negative tests (abuse tests) what causes failure Test outside the intended design, e.g., wrong configuration, wrong input, etc. Reveals weaknesses in design Stress tests show problems with insufficient or shared resources

  11. Testing Workflow Procedures • Specify how to perform one or several test cases or • parts of them • Instructions for tester in manual test case • Instructions for interacting with test automation tool to • create executable test components, then instructions to • integrate and execute those test components • One test procedure may cover several test cases • One test case may require several test procedures • Test procedure instructions often reflect flow-of events • descriptions, including input values, how to enter input • values, and how to verify results

  12. Testing Workflow Components Automate one or several test procedures or parts Provide test inputs, control and monitor test execution, and possibly analyze and report results Developed with scripting languages, programming languages, or test automation tools test drivers, test harnesses, and test scripts Complex test cases and procedures may require formal analysis, design, and implementation modeling of the test components

  13. Testing Workflow Plan Describes testing strategies, resources and schedule · Strategy What kind of tests to perform for each iteration Test objectives Required level of test and code coverage Test acceptance criteria Trade-off cost vs. improved quality and reduced risk Test Plan may be defined in conjunction with the model artifacts from other workflows analysis (black box) design and implementation (white box) “Tests” may include artifact reviews

  14. Testing Workflow Results Results are the output data of the test and/or the behavior of the system during test Analysis of results: pass or defect Defect: a system anomaly (software fault or a defect found in artifact review) capture symptom in sufficient detail to allow problem resolution capture defect in problem and resolution tracking system Test evaluation: summary of overall test results test-case coverage, code coverage defects and defect status

  15. Testing Workflow Results

  16. Testing Workflow Workers Test designer: responsible for integrity of the test model plan tests: test goals and schedule select and define test cases, test procedures evaluate test results Component engineer: responsible for development of test components that automate some of the test procedures consider independent test

  17. Testing Workflow Workers Integration tester: perform integration tests and report defects may be same person as system integrator in implementation workflow System tester: perform system tests and report defects focus on externally observable behavior consider users, beta customers, etc. as testers

  18. Testing Workflow – THE PLAN Plan the Tests Strategy and Resources Define Integration Tests Define System Tests Design Regression Tests Define and Mutate Test Cases Implement Tests Evaluate Tests

  19. Testing Workflow – THE PLAN Plan the Tests Strategy and Resources Describe testing strategy Inputs: use case model, supplemental requirements, design model Develop test cases and procedures that have a minimum of overlap test the most important test cases test high risk requirement items Estimate resources Estimate human and system resources Schedule the testing effort

  20. Testing Workflow – THE PLAN Define Integration Tests Derive test cases from use-case realizations – design (component interaction) Develop tests with minimal overlap that test an interesting path or scenario through use case Primarily consider interaction diagrams Combinations of actor input, output and system start state that lead to interesting scenarios that employ the classes that participate in the diagrams

  21. Testing Workflow – THE PLAN Define System Tests Test system as a whole Test combinations of use cases instantiated under different conditions Hardware configurations System loads Number of actors Database size, etc. Prioritize combinations required or are likely to function in parallel likely to influence each other if in parallel involve multiple processes frequently use system resources or use them in complex and unpredictable ways

  22. Testing Workflow – THE PLAN Design Regression Tests Modify test cases from earlier builds to be used as regression tests in subsequent builds Make more flexible and resilient to changes to the system under test Specifically address previously found defects Automate where possible

  23. Testing Workflow – THE PLAN Define and Mutate Test Cases Work test case by test case and suggest testing procedures for each one Reuse existing test cases May need to modify them to be reusable Align test procedures to subsystems under test Minimize overlap Isolates change to test resulting from change to subsystem

  24. Testing Workflow – THE PLAN Implement Tests Create test components from test procedures With test automation tools Perform or specify actions Record actions as a test component When programming explicitly Design and implement using test procedures Provide input and output data generation, visualization, interpretation and management utilities Use spreadsheets and database applications

  25. Testing Workflow – THE PLAN Evaluate Tests Perform the integration tests relevant to a build Manually perform test procedures Compare test results with the expected test results Investigate test results that deviate from the expected Faulty system and/or faulty test? Report defects to the component engineer responsible for the components likely to contain the fault Report defects to the test designers for evaluation of overall test results

  26. Testing Workflow – THE PLAN Evaluate Tests (cont) Test designers prepare metrics of test acceptance with respect to system quality level Test completeness and coverage Percentage of test cases executed Percentage of code and paths tested Reliability Categorize defect severity Analyze trends of discovered defects Compare with prior experience to predict amount of testing still required to reach an acceptable defect leve

  27. Testing Workflow – THE PLAN Evaluate Tests (cont) Suggest actions Suggest further tests Relax test criteria Deliver parts of the system that passed; rework and retest parts that failed Document test completeness, reliability and actions

  28. Testing Workflow – SUMMARY Test Model Test cases specify what to test Test procedures specify how to perform tests Test components automate test procedures Test Plan Test Evaluation Defects to be fed into other core work

More Related