1 / 11

Evolutionary Test Generation - Some Terminology

Evolutionary Test Generation - Some Terminology. Prof. Dr. Holger Schlingloff Humboldt-Universität zu Berlin and Fraunhofer Institute of Computer Architecture and Software Technology FIRST. Why “Evolutionary Test Generation”?.

mrinal
Download Presentation

Evolutionary Test Generation - Some Terminology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evolutionary Test Generation- Some Terminology Prof. Dr. Holger Schlingloff Humboldt-Universität zu Berlin and Fraunhofer Institute of Computer Architecture and Software Technology FIRST

  2. Why “Evolutionary Test Generation”? • Systematic testing is the most widely used quality assurance method • Effectiveness depends on the test suite • Manual test case construction is tedious and error-prone • semi-automatic procedures often achieve only insufficient coverage • Heuristic search methods needed • Genetic algorithms for complex search problems • Selection, mutation, and recombination of test suites • Problems • testing objectives • coverage and reliability measures • representation of test cases and test suites • seeding, recombination and mutation strategies • …

  3. What is „Software Testing“? • My definition: „The process of systematically experimenting with an object in order to establish its quality.“ • Experiment: singular activity to find something out • Probe: experiment to find out the quality • Test:systematic set of probes • Systematic = in the way in which the object (system) is composed • needs planning • needs analysis of the object or its specification • needs measurement

  4. Tested Quality What is „Software Quality“? • My definition: „The degree of accordance to the intention or specification“ • no absolute notion of quality • many possible quality measures • functionality, usefulness • efficiency (time, space, money, energy) • safety, reliability, robustness, fault-tolerance • usability, stability, maintainability, … • most important: correctness, i.e. absence of errors • intention or specification must be written down

  5. Testing, Validation, Verification Trying to answer different questions • Testing: Did we build the software right? • Validation: Did we build the right software? • Verification: Can we show that the software is correct? • Dijkstra: “Testing can only show the presence of bugs, not their absence.” • NN (attributed): “Beware of this program. I haven’t tried it yet, I only proved its correctness.”

  6. Error Fault Failure Errors, Faults, Failures • Error or Mistake – something a person thought or did he/she shouldn’t have (bad idea or action) • Fault or Defect – something wrong within the design or machine (bad state or flaw), due to an error during the design or manufacturing process • Failure – wrong behaviour, malfunction of an artefact due to the activation of a fault • Incident or Accident – visible effect of a failure onto the environment of the system, esp. on people

  7. Tests, Test Cases and Test Suites • Test – the execution of a test case • Test case – an entity identifying preconditions, inputs and expected outputs or postconditions for a particular SUT behaviour • Test suite – set of test cases for a particular testing objective (quality measure), usually with common points of observation and control (PCOs) in the SUT Test Case A34871 Table lookup mod_admin Pre: n_usr>0 In: uid = 0x5f0e Out: uname =“abc” Post: tbl=tbl’ • Testing objective – the purpose of tests

  8. Scenarios (A) unit testing of C programs (B) object-oriented testing of C++ / Java code (C) model-based black box testing (D) real time and hybrid systems testing (E) specification based testing (F) use-case testing (G) load, stress and efficiency testing (H) GUI testing • each of these scenarios has different testingobjectives and requires different kinds of test cases

  9. Test Design • Testing goal – a particular item in the SUT to be tested • a statement to be reached, a limit to be approached, a requirement to be tested • not to be confused with testing objective • Test design – the construction of test suites • Test generation – automatic or semi-automatic test design • Coverage – percentage of test goals reached by a test suite • statement coverage, input coverage, requirements coverage, …

  10. Test Execution • Testing is the activity of performing tests • executing – stimulating inputs, observing outputs of SUT • often this involves a test driver or SUT adapter • monitoring the progress of the test execution • evaluating the result of the tests (test oracle, verdict) • comparison between specified and realized functionality • Testing tools • test environment, harness, specification • testing languages • test management

  11. Evolutionary Terms • Chromosome / Genome / Individual = a test case (?) • Population / Generation = a test suite (?) • Fitness of an individual vs. fitness of a population • Fitness function = testing objective (?) • Selection and breeding = (in testing ?) • Genetic operator (Mutation / Recombination / Crossover) = (in testing ?)

More Related