Integration and system Testing

1. Integration and system Testing Csci565 Fall 2012 H.Reza

2. Objectives • Integration Testing • Simple ATM (SATM) • discuss integration testing strategies • Decomposition Based Integration Testing (DBIT) • Top-down • Bottom-up • Call Graph Based Integration Testing (CGBIT) • Path-Based Integration Testing (PBIT) H.Reza

3. A Software Testing Strategy: Spiral Model System Testing validation Testing Integration Testing Unit Testing Code Design Requirements System eng.

4. Integration Testing:1 • If all units/components work individually, do they work as a whole when we put them together? • Not necessarily • The problem is “putting them together” or interfacing them H.Reza

5. Problems with interfacing • Integration faults often traceable to incomplete or misunderstood interface specifications • mismatched assumptions about other components • Individually acceptable imprecision may be magnified to unacceptable levels • Global data structures can present problems • Inconsistent interpretation of parameters or values • Mixed units (meters/yards) in Martian Lander • Violations of value domains, capacity, or size limits • … H.Reza

6. Integration testing • Tests complete systems or subsystems composed of integrated components • Integration testing should be black-box testing when tests derived from the specification • Main difficulty is localising errors • Incremental integration testing reduces this problem H.Reza

7. Approaches to integration testing • Two major approaches • Incremental approaches • The decomposition-based techniques or tree • Stubs/Drivers • Call Graph-based techniques • No stubs or drivers • Non-incremental approaches H.Reza

8. Incremental integration testing H.Reza

9. Incremental approaches: Top-down • Top-down testing • Start with high-level system • integrate from the top-down replacing individual components by stubs where appropriate • Depth-first • Breadth-first • No-best order • Critical sections • Early skeletal version using I/O modules H.Reza

10. Stubs • Stubs • Special module to simulate some functionality • Its production is nontrivial task because the code may simulate a very complicated tasks • E.g. • Writing a stub performing a database table search routine • Creating multiple version of same stub for various reasons H.Reza

11. Top-down testing H.Reza

12. Top-down: Complications • The most common complication occurs when processing at low level hierarchy demands adequate testing of upper level • To overcome: • Either, delay many tests until stubs are replaced with actual modules (BAD) • Or, develop stubs that perform limited functions that simulate the actual module (GOOD) • Or, Integrate the software using bottom up approach H.Reza

13. Incremental testing: Bottom up • Bottom-up testing • Integrate individual components in levels until the complete system is created H.Reza

14. Bottom-up approach • Starts with construction and testing with atomic modules • No need for stub • Low level components are combined into cluster (or builds) to perform a specific sub-function • A driver (a control program for testing) is written • Contain hardcoded test input, calls the module being tested, and display the results • Cluster is tested • Drivers are removed and clusters are combined moving upward in the program structure H.Reza

15. Bottom-up testing M H.Reza

16. Guideline for Integration testing • Integrate the components that implement the most frequently used functionality • Perform regression testing for old features • Perform progression testing for new features • Study the Architecture design of the Application • identify the Critical Modules. • Obtain the interface designs from the Architectural team and create test cases to verify all of the interfaces in detail. • E.g., Interface to database/external hardware/software application must be tested in detail. H.Reza

17. Non-incremental • Big-bang • Imposes no order (GOOD) • Test all the units (Modules) at once (GOOD) • Very easy (GOOD) • difficult to localize the source of errors (VERY BAD) H.Reza

18. Top-down Vs. Bottom-up • Architectural validation • Top-down integration testing is better at discovering errors in the system architecture • System demonstration • Top-down integration testing allows a limited demonstration at an early stage in the development • Test implementation • Often easier with bottom-up integration testing H.Reza

19. Primary Design of the ATM using Tree-based decomposition Requirement specifications Terminal I/O Mange Session Conduct Transactions Select Transaction Card Entry PIN Entry H.Reza

20. Simple ATM (SATM) • An ATM simple • Provides 15 screens for interactions • includes 3 function buttons • B1, B2, B3 • Modeled in structural analysis • Data Model (ERD) • Functional Model (DFD) • Behavioral model (STD) H.Reza

21. Figure 12.7 H.Reza

22. Figure 12.8 H.Reza

23. Figure 12.9 H.Reza

24. Figure 12.10 H.Reza

25. Entity/Relationship model of the SATM

26. Figure 12.12 H.Reza

27. Figure 12.13 H.Reza

28. Decomposition based strategies • Decomposition based • Top/down • Bottom up • Sandwich • Big bang H.Reza

29. Figure 12.14 H.Reza

30. PDL description of SATM H.Reza

31. H.Reza

32. H.Reza

33. Figure 13.1 H.Reza

34. Decomposition based testing:1 • Discussion revolves around the tree-based decomposition and the order by which units are tested and combined • Top-to-bottom • Bottom-to-top • Sandwich • Big bang • The focus is on the structural compatibility among interfaces H.Reza

35. Test Sessions • A test session refers to one set of tests for a specific configuration of actual code and stubs • The number of integration test sessions using a decomposition tree can be computed • Sessions=nodes – leaves + edges H.Reza

36. decomposition based testing: 2 • For SATM system • 42 integration testing session (i.e., 42 separate sets of integration test cases) • top/down • (Nodes-1) stubs are needed • 32 stub in SATM • bottom/up • (Nodes-leaves) of drivers are needed • 10 drivers in SATM H.Reza

37. Decomposition based strategies: Pros and con • Intuitively clear and understandable • In case of faults, most recently added units are suspected ones • Can be tracked against decomposition tree • Suggests breadth-first or depth-first traversals • Units are merged using the decomposition tree • Implies correct behavior should follow from individually correct units and interfaces • Stubs/Drives are major development Overhead H.Reza

38. Call graph based integration testing • Call graph • A directed graph • Nodes corresponds to unit • Edges corresponds to the call • E.g. • AB (i.e., A is calling B) • Attempts to overcome the decomposition problem (structural) • Moves toward behavioral testing H.Reza

39. Call graph based(CG): approaches • Two main approaches based on Call Graph • Pair-wise integration • Neighborhood integration H.Reza

41. Table 2: AM H.Reza

42. Pair-wise integration • The main idea is to eliminate the overhead (i.e., stub/drive) • Uses actual code by restricting a session testing to a pair of units in the Call Graph • One integration test for each edge in Call Graph • 40 edges means 40 integration tests for the SATM H.Reza

43. Pairwise Uses actual code -one integration test session for each edge -40 edges for SATM H.Reza

44. Neighborhood Integration • The neighborhood of a node refers to the nodes that are one edge away from the given nodes • SATM Neighborhoods • Number of neighborhoods can be computed: • 1) Neighborhoods = Interior node + source node • 2) Interior nodes =nodes – (source nodes + sink nodes) • ( x=1 if there exists leaf nodes connected directly to the root node otherwise X= 0) • Results a drastic reduction in the number of integration test session • In case of SATM (11 vs. 40) H.Reza

45. Neighborhood Integration H.Reza

46. Table 3: SATM Neighborhoods H.Reza

47. Pros and cons • Benefits (GOOD) • Mostly behavioral than structural • Eliminates sub/drive overhead • Works well with incremental development method such as Build and composition • Liabilities (BAD) • The fault isolation • E.g., • Fault in one node appearing in several neighborhood H.Reza

48. Questions about Integration testing • How do perform Integration testing for non-tree based functional decomposition? • E.g., • integration testing for OO • Integration testing for Client/server systems • Integration testing for Layered systems • …. H.Reza

49. Path-based Integration Testing • The hybrid approach (i.e., structural and behavioral) is an ideal one integration testing • The focus is on the interactions among the units • Interfaces are structural • Interactions are behavioral • With unit testing, some path of source statements is traversed • What happens when there is a call to another unit? • Ignore the single-entry/single-exit • Use exist follows by an entry sequence • Suppress the call statement H.Reza

50. New and extended concepts • Source node (begin) • A statement fragment at which program execution begins or resumes • E.g., BEGIN • Sink node (end) • A statement fragment at which program execution terminates • E.g., END H.Reza