1 / 34

Automated Test Data Generation for Aspect-Oriented Programs

Automated Test Data Generation for Aspect-Oriented Programs. Mark Harman (King’s College London, UK) Fayezin Islam (T-Zero Processing Services, US) Tao Xie (North Carolina State University, US) Stefan Wappler (Berner & Mattner, Germany). Background.

sibley
Download Presentation

Automated Test Data Generation for Aspect-Oriented Programs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Automated Test Data Generation for Aspect-Oriented Programs Mark Harman (King’s College London, UK) Fayezin Islam (T-Zero Processing Services, US) Tao Xie(North Carolina State University, US) Stefan Wappler (Berner & Mattner, Germany)

  2. Background Automated testing of aspect-oriented programs Testing aspectual composition behavior (pointcut behavior) [Ferrari et al. ICST 08, Anbalagan&Xie ISSRE 08, …] Testing aspectual behavior (advice behavior) [Xie&Zhao AOSD 06, Xie et al. ISSRE 06, ...]

  3. Testing Aspectual Behavior Aspect weaving (e.g., with ajc, abc) Aspect  Class (bytecode) Advice  Method (bytecode) Straightforward unit testing: feed aspect classes to OO test generation tools based on bytecode Issues: arguments can be thisJoinPoint or AroundClosure objects Aspectra: generate tests for woven classes but focus on aspectual behavior Feed woven classes to OO test generation tools Base classes == “test drivers” Leverage existing OO tools for testing AOP programs [Xie&Zhao AOSD 05] E.g., Parasoft Jtest (random testing tool)

  4. Example Program Under Test +

  5. New Contributions • A new system of automated test data generation for AOP based on Search-Based Testing, i.e., evolutionary testing • Input domain: receiver object, method parameters • E.g., account.debit(amount) • Empirical studies to demonstrate the benefits of the system in AOP structural testing • Effectiveness: better than random testing • Efficiency: AOP domain reduction techniques • Efficiency: focusing test effort on aspectual branches to improve efficiency

  6. What is Search-Based Testing? In search-based testing, we apply search techniques to search large input spaces, guided by a fitness function. Fitness function measures how good/close one input is in reaching the goal e.g., covering true branch of

  7. Evolutionary Algorithms chromosome Insertion Mutation Fitness evaluation Recombination End? Selection

  8. Evolutionary Testing Insertion Test cases Mutation Fitness evaluation Recombination End? Selection

  9. Evolutionary Testing Insertion Test cases Mutation Fitness evaluation Execution Recombination End? Selection

  10. Evolutionary Testing chromosome Insertion Method seq Test cases Mutation Fitness evaluation Execution Recombination Monitoring End? Selection

  11. Structural Evolutionary Testing Structural Evolutionary Testing Target:true branch

  12. Evaluation of predicate in a branching condition if (A < B) Fitness = (A - B) + 1 101 = (100 – 0) + 1 51 = (100 – 50) + 1 0 = (100 – 101) + 1 Structural Evolutionary Testing Structural Evolutionary Testing Lower fitness value, the better 0 fitness value, reach the target Target:true branch

  13. AOP Domain Reduction - Motivation • Input domain: receiver object, method parameters • E.g., account.debit(amount) • Not all input variables are relevant to the coverage of the target branch inside aspects Target:false branch …

  14. Slicing-based Domain Reduction • Irrelevant-input-variable identification • Start with slicing criterion: predicates of target branches • Extract backward program slices (based on data and control dependence) • Identify relevant input variables: input variables showing up in the slices: account.debit(amount) • Domain reduction: searching for only relevant input vars Target:false branch …

  15. EvolutionaryAspectTester (EAT) System Implementation • Indus slicer [Ranganath et al. 07] Aspectra [Xie&Zhao 06] • EvoUnit [Wappler 08]

  16. Evaluation Benchmarks 14 benchmarks from [Xie&Zhao 06, Rinard et al. 04, Dufour et al. 04, Hannemann&Kiczales 02]

  17. Study 1: Assessment of evo testing RQ 1.1. Can evolutionary testing outperform random testing for AOP testing?

  18. RQ 1.1: Assessment of evo testing 43% Coverage improvement of evolutionary testing over random testing Better branch coverage on 5/14 benchmarks

  19. RQ 1.1: Assessment of evo testing cont. 61% Effort reduction of evolutionary testing over random testing Effort reductionon 9/14 benchmarks

  20. Findings: Assessment of evo testing • RQ 1.1. Can evolutionary testing outperform random testing for testing aspect-oriented programs? • Better branch coverage on 5/14 benchmarks • (0%~43%) • Effort reductionon 9/14 benchmarks • (0%~61%)

  21. Study 2: Impact of domain reduction RQ 2.1. #branches that have irrelevant parameters and %parameters that are irrelevant for each such branch? RQ 2.2. %effort reduction for each such branch? RQ 2.3. %effort reduction for each program?

  22. RQ 2.1: Impact of domain reduction 90/434 aspectual branches with irrelevant parameters

  23. RQ 2.1: Impact of domain reduction Input domain reduction for branches with non-0 reduction Input domain reduction (25%~100%)

  24. RQ 2.2: Impact of domain reduction 94% Easy/trivial branches -88% Effort reduction per branch of using domain reduction Effort increase on 25%, same on 6%, and reduction on 69% branches

  25. RQ 2.3: Impact of domain reduction Effort reduction per program of using domain reduction Effort reduction (17%~93%)

  26. Findings: Impact of domain reduction RQ 2.1. #branches that have irrelevant parameters (99/434) and %parameters that are irrelevant for each such branch (25%~100%)? RQ 2.2. %effort reduction for each such branch (-88%~94%)(69% branches get reduction)? RQ 2.3. %effort reduction for each program (17%~93%)?

  27. Study 3: Impact of focusing on testing aspectual behavior RQ 3.1. %effort reduction for test data generation if aspectual behavior instead of all behavior is focused on?

  28. RQ 3.1: Impact of aspect focusing 99.99% 3% Effort reduction of focusing on aspectual behavior over all behavior Effort reduction on all 14 benchmarks

  29. RQ 3.1: Impact of aspect focusing cont. 62% Coverage improvement of focusing on aspectual behavior over all behavior Coverage improvement on 6/14 benchmarks

  30. Study 3: Impact of focusing on testing aspectual behavior • RQ 3.1. %effort reduction for test data generation if aspectual behavior instead of all behavior is focused on? • Effort reduction on all 14 benchmarks • (3% ~ 99.99%) • Coverage improvement on 6/14 benchmarks • (0% ~ 62%)

  31. Conclusion • A new system of automated test data generation for AOP based on Search-Based Testing • Empirical studies to demonstrate the benefits of the system in AOP structural testing • Effectiveness: better than random testing • Efficiency: AOP domain reduction techniques • Efficiency: focusing test effort on aspectual branches to improve efficiency • Future work on more advanced techniques (e.g., symbolic execution), more testing objectives, larger AOP programs

  32. Questions?

  33. Identify relevant branching statements using control dependence, e.g.,(#expected/#actual – 1) Approximation level Level 4 Level 3 Evaluation of predicate in a branching condition if (A < B) Local_Distance = (A – B) + 1 Level 2 Fitness = Approximation_Level + Local_Distance 101 = 0 + (100 – 0) + 1 51 = 0 + (100 – 50) + 1 0 = 0 + (100 – 101) + 1 Level 1 Target Structural Evolutionary Testing Structural Evolutionary Testing Local distance Lower fitness value, the better 0 fitness value, reach the target

  34. RQ 2.4: Impact of domain reduction Co-lateral coverage improvement effect of domain reduction 9 branches have statistically significant change in co-lateral coverage

More Related