1 / 37

Object-Oriented and Classical Software Engineering Fifth Edition, WCB/McGraw-Hill, 2002 Stephen R. Schach srs@vuse.vand

Object-Oriented and Classical Software Engineering Fifth Edition, WCB/McGraw-Hill, 2002 Stephen R. Schach srs@vuse.vanderbilt.edu. CHAPTER 6. TESTING. Overview. Quality issues Nonexecution-based testing Execution-based testing What should be tested? Testing versus correctness proofs

zavad
Download Presentation

Object-Oriented and Classical Software Engineering Fifth Edition, WCB/McGraw-Hill, 2002 Stephen R. Schach srs@vuse.vand

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Object-Oriented and Classical Software EngineeringFifth Edition, WCB/McGraw-Hill, 2002Stephen R. Schachsrs@vuse.vanderbilt.edu

  2. CHAPTER 6 TESTING

  3. Overview • Quality issues • Nonexecution-based testing • Execution-based testing • What should be tested? • Testing versus correctness proofs • Who should perform execution-based testing? • When testing stops

  4. Testing • Two types of testing • Execution-based testing • Nonexecution-based testing

  5. Testing (contd) • “V & V” • Verification • Determine if the phase was completed correctly • Validation • Determine if the product as a whole satisfies its requirements

  6. Testing (contd) • Warning • “Verify” also used for all nonexecution-based testing

  7. Software Quality • Not “excellence” • Extent to which software satisfies its specifications • Software Quality Assurance (SQA) • Goes far beyond V & V • Managerial independence • development group • SQA group

  8. Nonexecution-Based Testing • Underlying principles • We should not review our own work • Group synergy

  9. Walkthroughs • 4–6 members, chaired by SQA • Preparation—lists of items • Inspection • Up to 2 hours • Detect, don’t correct • Document-driven, not participant-driven • Verbalization leads to fault finding • Performance appraisal

  10. Inspections • Five-stage process • Overview • Preparation, aided by statistics of fault types • Inspection • Rework • Follow-up

  11. Fault statistics • Recorded by severity and fault type • Compare with previous products • What if there are a disproportionate number of faults in a module? • Carry forward fault statistics to the next phase

  12. Statistics on Inspections • 82% of all detected faults (IBM, 1976) • 70% of all detected faults (IBM, 1978) • 93% of all detected faults (IBM, 1986) • 90% decrease in cost of detecting fault (Switching system, 1986) • 4 major faults, 14 minor faults per 2 hours (JPL, 1990). Savings of $25,000 per inspection • Number of faults decreased exponentially by phase (JPL, 1992) • Warning • Fault statistics and performance appraisal

  13. Metrics for Inspections • Fault density (e.g., faults per KLOC) • Fault detection rate (e.g., faults detected per hour) • By severity (major/minor), by phase • What does a 50% increase in the fault detection rate mean?

  14. Execution-Based Testing • Definitions • Failure (incorrect behavior) • Fault (NOT “bug”) • Error (mistake made by programmer) • Nonsensical statement • “Testing is demonstration that faults are not present”

  15. What is execution-based tested? • “The process of inferring certain behavioral properties of product based, in part, on results of executing product in known environment with selected inputs.” • Inference • Known environment • Selected inputs • But what should be tested?

  16. Utility • Does it meet user’s needs? • Ease of use • Useful functions • Cost-effectiveness

  17. Reliability • Frequency and criticality of failure • Mean time between failures • Mean time to repair • Mean time, cost to repair results of failure

  18. Robustness • Range of operating conditions • Possibility of unacceptable results with valid input • Effect of invalid input

  19. Performance • Extent to which space and time constraints are met • Real-time software

  20. Correctness of specifications • Incorrect specification for a sort • Function trickSort which satisfies this specification:

  21. Correctness of specifications (coptd) • Incorrect specification for a sort: • Corrected specification for the sort:

  22. Correctness • NOT necessary • NOT sufficient

  23. Correctness Proofs • Alternative to execution-based testing

  24. Example of Correctness Proof • Code segment to be proven correct

  25. Example of Correctness Proof (contd) • Flowchart of code segment

  26. Example of Correctness Proof

  27. Correctness Proof Case Study • Never prove a program correct without testing it as well

  28. Episode 1 • 1969 — Naur Paper • “Naur text-processing problem” Given a text consisting of words separated byblankor bynl (new line) characters, convert it to line-by-line form in accordance with following rules: (1) line breaks must be made only where given text has blankor nl ; (2) each line is filled as far as possible, as long as (3) no line will contain more than maxpos characters • Naur constructed a procedure (25 lines of Algol 60), and informally proved its correctness

  29. Episode 2 • 1970 — Reviewer in Computing Reviews • The first word of the first line is preceded by blank unless the first word is exactly maxpos characters long

  30. Episode 3 • 1971 — London finds 3 more faults • Including: • The procedure does not terminate unless a word longer than maxpos characters is encountered

  31. Episode 4 • 1975 — Goodenough and Gerhart find three further faults • Including: • The last word will not be output unless it is followed by blank or nl

  32. Proofs and Software Engineering • Lesson: • Even if product is proved correct, it must STILL be tested.

  33. Three Myths • Software engineers do not have enough math for proofs • Proving is too expensive to be practical • Proving is too hard

  34. Proofs and Software Engineering (contd) • Can we trust a theorem prover ? • How to find input–output specifications, loop invariants • What if the specifications are wrong? • Can never be sure that specifications or a verification system are correct

  35. Proofs and Software Engineering (contd) • Correctness proofs are a vital software engineering tool, WHERE APPROPRIATE. • If • Human lives are at stake • Indicated by cost/benefit analysis • Risk of not proving is too great • Also, informal proofs can improve the quality of the product • Assert statement

  36. Who Performs Execution-Based Testing? • Testing is destructive • A successful test finds a fault • Solution • 1. The programmer does informal testing • 2. SQA does systematic testing • 3. The programmer debugs the module • All test cases must be • Planned beforehand, including expected output • Retained afterwards

  37. When Testing Can Stop • Only when the product has been irrevocably retired

More Related