SIMULATING ERRORS IN WEB SERVICES

1. SIMULATING ERRORS IN WEB SERVICES International Journal of Simulation: Systems, Sciences and Technology 2004 Nik Looker, Malcolm Munro and Jie Xu

2. Outline • Introduction • Quality of Service • Fault Injection • Fault Model • Test Case • Conclusion and Future Work

3. Introduction • Web Service technology is a key factor in the success of any e-Science or e-Commerce project. • High Quality of Service (QoS) to allow the production of highly reliable systems. • Fault injection is a well-proven method of assessing the dependability of a system

4. Introduction • This fault injection method is a modified version of Network Level Fault Injection. • This method differs from standard Network Level Fault Injection techniques • Fault injector decodes the SOAP messages and can inject meaningful faults into a message • This enables API-level parameter value modification to be performed as well as standard Network Level Fault Injection.

5. Quality of Service • Availability • the quality aspect of whether a Web Service is present and ready for use. • Accessibility • the quality aspect that represents the degree the Web Service is capable of servicing a request. • Integrity • the quality of the Web Service maintaining the correctness of any interaction. • Performance • the quality aspect that is defined in terms of the throughput of a Web Service and the latency.

6. Quality of Service • Reliability • the quality aspect that represents the capability of maintaining the service and service quality. • Regulatory • the quality aspect that the service corresponds to rules, laws, standards and specifications. • Security • the quality aspect that defines confidentiality for parties using a service

7. Fault Injection Techniques • Network Level Fault Injection • Corruption, loss or reordering of network messages at the network interface. • Runtime injection technique that injects faults into captured network packets. • Software Implemented Fault Injection (SWIFI) • instrumenting the operating system protocol stack • Perturbation • Modify program states without mutating existing code statements. • testing fault tolerance mechanisms

8. Fault Injection Techniques • Corruption of Memory Space • Platform Specific • Code Mutation • Certification Problems because code changed • Syscall Interposition • Platform Specific

9. Fault Injection Tools • Ferrari (Fault and Error Automatic Real-Time Injection) • uses a system based around software traps to inject errors into a system. • traps are activated by either a call to a specific memory location or a timeout. • FTAPE (Fault Tolerance and Performance Evaluator) • inject faults into memory, registers and disk accesses

10. Fault Injection Tools • Xception • processor’s exception handling capabilities are used to trigger fault injection. • DOCTOR (IntegrateD sOftware Fault InjeCTiOn EnviRonment) • allows injection of memory and register faults, and network communication faults. • combination of timeout, trap and code modification.

11. Fault Injection Tools • Orchestra • a script driven fault injector which is based around Network Level Fault Injection.

12. WS-FIT • Web Service – Fault Injection Technology • Method and tools implement mechanism of fault injection and also provide a framework for the creation and execution of test cases • Fault injector that allows network level fault injection • Dependability Assessment of SOAP based SOA • SOAP Based • WSDL Defined Interfaces • Combined with conventional specification

13. Fault Model • Fault Types that can effect a SOAP based system: • Physical Faults • Effecting memory or processor registers • Software Faults • Programming errors and design errors • Resource Management Faults • Memory leakage and exhaustion of resource • Communication Faults • Message deletion, duplication, delay, reordering or corruption • Life-Cycle Faults • Premature object destruction through starvation of keep-alive messages and delayed asynchronous responses

14. Enhanced Fault Model

15. Failure Modes • Crash of a service • Crash of a hosting web server (or the host itself) • Hang of a service • Corruption of input data • Corruption of output data • Duplication of messages • Omission of messages • Delay of messages

16. Why WS-FIT testing? • Particularly interested in • Assessing Quality of Service • Assessing Fault Tolerance of Systems • Developing Methods and Tools • So that we can • Detect defects in platform code and design • Gather dependability metrics on platforms to allow comparisons • Use WS-FIT to test individual Web Services

17. WS-FIT • Consists of 3 parts: • Hook code in SOAP Library • Fault Injector Framework • Script • Trigger • Injection • Results

18. SOAP API Hooks • SOAP API Hooks • One for sending messages. • One for receiving messages. • Hooks consisted of simple socket code • To pass messages to fault injector. • Receive (possibly) modified message from injector.

19. Instrumenting a Service

20. What the Framework Provides • Logging Function (XML format for easy analysis) • Detects faults and logs faults • Logs injected fault packets • Trigger and Injection mechanism: • Trigger on Message Type and parameter. • Inject Fault through User Script. • Implemented • Framework implemented in Java for portability • Scripts implemented in Python • Executed in the JVM by Jython

21. User Script • Result class • One instance of this for duration of test. • Used to store static results. • Currently only tag count but will be enhanced in later experiments. • Trigger Class • One instance per packet. • Returns Boolean value to indicate if injector class should be run. • Use data from result class to determine if the correct point in message stream has been reached. • Injector Class • One instance per packet • Inject fault into packet

22. WS-FIT

23. GUI • The GUI provides • An easy to use environment to create and execute tests • Ability to create a skeleton test script from WSDL definitions • Populate tests through user defined Fault Model • Execution of test scripts from tool • Real-time visualization of RPC message parameters

24. Quality of Service • Experiment to demonstrate WS-FIT can be used to assess QoS criteria • WS-FIT will be used to modify the performance of a system by injecting a latency (without code modification being required)

25. Test Scenario • The test system implements a simple, self regulating, heater unit. • The hardware used is simple and relies on the software drivers to prevent failure states being reached. • Under normal operation the ‘Unit’ will give a logarithmic temperature rise so the Controller uses timers to modify to a linear temperature rise. • Will be used in a chemical experiment • A substance must be heated to 60°C • Must reach 60ºC in a specified period of time • Must have a linear temperature rise • The test will introduce a latency to decrease the throughput and hence the performance of the system.

26. Test System

27. Natural Operation of Heater

28. Model

29. Applying a Fault Model to aParameter

30. Injecting Latency

31. Summary • WS-FIT can be effectively used to: • Introduce SOAP protocol faults • Introduce both random and non-random protocol faults • Assess Quality of Service • Such as throughput modification • WS-FIT • Relatively non-invasive • Can be used to assess both middleware and individual services • Gives real-time feedback through visualization of RPC parameters http://www.wsfit.org/