Software testing Lecture p8 T120B029 20 12 pavasario sem.

1. Software testing Lecture p8T120B0292012 pavasario sem.

2. Motivation “ Testing is the process of executing a program with the intent of finding errors”

3. Limits of software testing • “Good” testing will find bugs • “Good” testing is based on requirements,i.e. testing tries to find differences between the expected and the observed behavior of systems or their components • BUT: Testing can only prove the presence of bugs - never their absence

4. Test Specification - Manual Process Test Cases Test Document Inspection Manual Manual Manual Text-based

5. Semi-Automated Process Test Cases Test Document Inspection Semi- Automated Automated Manual Model-based Consistency SyntaxSemantic

6. The V-model Requirements Definition I Acceptance Level Testing Functional system Design System Level Testing II Construction phasess Testing phasess III Integration Level Testing Technical system Design IV Component Specification Unit Level Testing Implementation

7. The W-model I Preparation of Acceptance Level Testing Acceptance Level Testing Debuging Preparation of System Level Testing System Level Testing II Preparation of Integration Level Testing Construction phasess Integration Level Testing III Preparation of Unit Level Testing IV Unit Level Testing Implementation

8. Using UML and UTP in system development UML UTP TTCN Debuging I II UML UTP TTCN Debuging UML Junit&TTCN III UTP Debuging IV UML UTP JUnit Debuging Implementation Changes

9. loop < 20x Exhaustive testing There are 1014 possible paths! If we execute one test per millisecond, it would take 3.170 years to test this program!!  Out of question

10. Testability = how easy can a program be tested Features that help/influence testing: • operability - it operates cleanly • observability • the results are easy to see • distinct output is generated for each input • incorrect output is easily identified • controllability • processing can be controlled • tests can be automated & reproduced • decomposability - software modules can be tested independently • simplicity - no complex architecture and logic • stability - few changes are requested during testing • understandability - program is easy to understand

11. developer independenttester Who tests the software Understands the system but, will test “gently” and, is driven by “delivery” Must learn about the system, but, will attempt to break it and, is driven by quality

12. Test cases • Describe how to test a system/module/function • Description must identify • system state before executing the test • function to be tested • parameter values for the test • expected outcome of the test  Some automation should be possible after defining them!

13. requirements output input events Black box vs. white box testing

14. White box testing …our goal is to ensure that all statements and conditions have been executed at least once...

15. White box testing:Why cover all paths? • logic errors and incorrect assumptions are inversely proportional to a path’s execution probability • we often believe that a path is not likely to be executed; in fact, reality is often counter intuitive • typographical errors are random; it’s likely that untested paths will contain some

16. If-then-else loop < 20x Guaranteed cover:Exhaustive testing But: There are 520=1014 (approx.) possible paths!

17. loop < = 20x Better:Selective testing a selected path

18. Path coverage testing 1) Derive a logical complexity measure 2) Use it to define a basis set of execution paths First, we compute the cyclomatic complexity V(G): number of transitions - number of nodes +2 In this case, V (G) = 4 V(G) provides an upper bound of tests that must be executed to guarantee coverage of all program statements

19. Path coverage set • Cyclomatic complexity defines the number of independent paths in the basis set • Path coverage set = set of paths that will execute all statements and all conditions in a program at least once • Goal: Define test cases for basis set • Path coverage set is not unique!

20. 1 2 4 3 5 6 7 8 Path coverage testing Next, we derive the independent paths: Since V(G) = 4, there are four paths Path 1: 1,2,3,6,7,8 Path 2: 1,2,3,5,7,8 Path 3: 1,2,4,7,8 Path 4: 1,2,4,7,2,4…7,8 Finally, we derive test cases to exercise these paths, i.e. choose inputs that lead to traversing the paths

21. Documenting test cases • Name • Number • Description of system state before running the test case • Values for the inputs • Expected outputs • Short description (if needed)

22. Path coverage testing - remarks • you don’t need a flow chart, but the picture will help when you trace program paths • each connection between boxes counts as 1 transition • path coverage testing should be applied to critical modules Connects four boxes  counts as three transitions

23. Black box testing requirements output input events

24. Categories of errors in black box testing • incorrect or missing functions • interface errors • errors in data structures or external database access • performance errors • initialization and termination errors

25. Example abs(integer x)Return x if x >= 0Return -x if x<0 • The concrete value of x does not matter! Basis for Equivalence Testing

26. Loops • Cornerstone of every program • Loops can lead to non-terminating programs

27. Loop testing simple loop nested loops concatenated loops unstructured loops

28. Loop testing: simple loops Minimum conditions - simple loops 1. skip the loop entirely 2. only one pass through the loop 3. two passes through the loop 4. m passes through the loop m < n 5. (n-1), n, and (n+1) passes through the loop where n is the maximum number of allowable passes

29. Nested loops • Just extending simple loop testing:  number of tests explodes • Reduce the number of tests: • start at the innermost loop; set all other loops to minimum values • conduct simple loop test; add out of range or excluded values • work outwards while keeping inner nested loops to typical values • continue until all loops have been tested

30. Equivalence class testing Equivalence partitioning mouse picks on menu output format requests user queries responses to prompts numerical data command key input Partitioning is based on input conditions

31. Equivalence classes (1) • Valid data • user supplied commands • responses to system prompts • file names • computational dataphysical parametersbounding valuesinitiation values • output data formatting commands • responses to error messages • graphical data (e.g., mouse picks)

32. Equivalence classes (2) • Invalid data • data outside bounds of the program • physically impossible data • proper value supplied in wrong place

33. Defining equivalence classes • Input condition is a range: one valid and two invalid classes are defined • Input condition requires specific value: one valid and two invalid classes are defined • Input condition is boolean: one valid and one invalid class are defined  Then define one test case for each equivalence class

34. Automating software testing • Manual software testing is time consuming • Software testing has to be repeated after every change (regression testing) • Write test drivers that can run automatically and produce a test report

35. Testing OO software: general remarks • Based on UML specifications • Use cases • Class diagrams • State transition diagrams • … • Problem: Effort focus on requirements not comprehensive  quality?

36. Scenario based testing • Concentrates on (functional) requirements • Based on • use cases • corresponding sequence diagrams

37. Test Requirements • Every use case • Every fully expanded extension (<<extend>>) combination • Every fully expanded uses (<<uses>>) combination • Tests normal as well as exceptional behavior

38. Example

39. Scenarios for ATM Example

40. Test Procedure (1) • Establish testing budget • Rank Use Cases (& variants) according to • Relative frequencies • Criticality • Allocate #test cases to each use case (and possibly variants) • Develop test cases for scenarios

41. Testing a use case/scenarios (1) • A scenario is a path through a sequence diagram • There are many different scenarios associated with a sequence diagram!

42. What can be wrong? • Incorrect or missing output • Missing function/feature in an object • Incorrect parameter values  boundary value analysis • Correct message - wrong object • Incorrect message - right object • Incorrect object state • Message sent to destroyed object • Incorrect exception thrown • Incorrect exception handling • Incorrect assumption about receiver’s class • Class casts

43. Testing a use case/scenarios (2) • All paths in sequence diagram should be executed • Focus on messages from the actor/UI to the system • If possible: check “internal” objects • Extract control flow information from a sequence diagram • Test branches and loops • Test exceptions • Consider state

44. <4] Example: Establish Session

45. Example (cont.) • Several checks are performed • Is the card an ATM card? • Is the card stolen? • Has the customer a valid bank account? • Has the customer a valid PIN? • Three chances to enter the PIN • Service charge if from different bank

46. Test Procedure (2) • Translate the sequence diagram into a flow graph • Each message or group of consecutive messages becomes a segment (box). • A conditional action (branch, iteration) ends a segment • Each bracketed (loop) condition becomes a decision (hexagon)

47. Establish Session

48. Remarks • White-box testing can be applied to flow graph • Control flow graph may reveal • ambiguities and omissions in the sequence diagram • Ambiguities either reveal faults or a behavior to be interpreted by the coder

49. Path conditions • Input and object state need to satisfy the path conditions • Identify predicates on each path, work from first predicate and identify required input and state variable values • Do the same for the next path conditions until the last one • Some paths may not be feasible

50. Example: Path Conditions