System Wide Information Management (SWIM)

1. System Wide Information Management (SWIM) Service-Oriented Architecture (SOA) Testing

2. Agenda • Traditional Testing • Migration to SOA • Conceptual SOA • Traditional vs. SOA Testing • SOA Testing Solutions • Anticipated Challenges

3. Characteristics of Traditional Testing • In most cases, non-agile but well established proven process • Single-system focused: not dependant on other systems within your business domain • Tightly coupled • Well-defined interfaces specified in the Interface Control Document (ICD)and Initial Requirements Document (IRD) • Limited security constraints, if any • Test program is based on the common Verification & Validation (V&V) test model • Test team only needs to be technically savvy on the system under test, and not the entire business domain • Test Tool Strategy is needed because systems are usually furnished with custom tools • Test requirements only apply to the system/subsystem and interfaces

4. Stand Alone Server Architecture • Applications share data and functions on an isolated single machine, no loose coupling • Each application communicates with others in a point-to-point, one off interface design • Difficult to upgrade or replace

5. Client Server Architecture • Server applications share data and functions on a single machine, no loose coupling • Each server based application communicates with others in a point-to-point, one-off interface design • Supports multiple users

6. Distributed Server Architecture • Server applications share data and functions across multiple server machines, possible loose coupling • Each server based application communicates with others in a point-to-point, intranet, or internet design • Supports multiple users and supplies composite data

7. Web Services Server Architecture • Server applications share data and functions across multiple server machines, possible loose coupling • External data and/or function provided externally • Each server based application communicates with others in a point-to-point, intranet, or internet design • Supports multiple users, supplies composite data

8. SOA • Data collections and function hosted by internal and external servers loosely coupled • Collections often communicate using a single interface • Reuse and upgrades are less challenging • Multiple remote users

9. SoftwareComplexityTimeline Test Complexity Year 1980 1990 2000 2010

10. TheConceptofSOA From Oasis (www.oasis-open.org), SOA is defined as – “An architectural style whose goal is to achieve loose coupling among interacting software agents/services.” SOA enables business flexibility in an interoperable, technology-agnostic manner. It consists of a composite set of business-aligned services that support a flexible and dynamically re-configurable end-to-end business processes realization using interface-based service description. SOA is a commonly chosen implementation architecture process for today's largely distributed businesses or systems.

11. Some Benefits of SOA are: • Reducing integration expense: uses open standards, not expensive proprietary middleware • Increasing asset reuse: legacy systems can be easily wrapped as services and exposed • Increasing business agility: services and systems may be rapidly developed, tested, and delivered • Reduction of business risk • Loosely coupled: a modified service does not affect other service or client • Interoperable: platform-neutral

12. Components that make up SOA • Service provider • Service consumer • Service registry • Governance Lead • Administrator SOA services are intended to be reusable and can be shared within the entire business domain

13. SOA Characteristics • SOA transport protocols: • Hypertext Transfer Protocol (HTTP) • HTTP Secure (HTTPS) • Java Message Service (JMS) • Messaging: • Extensible Markup Language (XML) • Web Services (WS)-Addressing • SOAP • Interface: • Web Service Description Language (WSDL) • Quality of Service (QoS): • Security • Reliable messaging • AuthN / AuthZ • Policies • Etc. • Governance: • National Airspace System (NAS) Service Registry/Repository (NSRR)

14. Common SOA Testing Challenges • No visibility below the User Interface to isolate errors • Data driven business logic is service embedded • Inaccessible external services • Inability to test components that do not have an available User Interface • Need for continuous validation of application functionality • Inability to test composite solutions due to limited access or availability of dependent services and data needed for testing • Inadequate or incomplete testing, resulting in costly problem identification and debugging once released into production

15. Common SOA Testing Challenges (cont.) • Poor collaboration between development and Quality Assurance (QA), with minimal to no reuse of test assets between unit, functional, regression and performance testing • Security testing needs to intensify as threat grows • Test team structure will require alignment to business domains (processes) and not to technologies to ensure that agility and speed on SOA testing do not compromise overall service quality • Organizations will have to develop and maintain “Test Assets” for key services to guarantee performance and security throughout the entire test coverage cycles

16. Traditional Testing vs. SOA Testing Similarities: • Both Traditional and SOA methodologies can utilize the success proven V-Model • The V-Model enforces testing discipline throughout the project life cycle • Both can utilize the top down or bottom up test approach

17. Differences: Traditional Testing vs. SOA Testing

18. Traditional Testing vs. SOA Testing Differences continued:

19. SOA Testing Solutions • Service Level Testing (SLT) emphasis • Security Testing and the project lifecycle • SOA and NAS experts leverage • Test Tool Strategy implementation

20. SLT Emphasis • SLT Emphasis • Test emphasis at the SLT level over Governance testing, service-component level testing, integration-level testing, process/orchestration-level testing, system-level testing, and Security testing • Formal Code Peer reviews • Ensures standards and compliance are met • Identifies potential interoperability, performance, and security defects and weaknesses • Continuous Functional, Performance and Security tests against the Services • Requires the use of test tools and/or test harnesses • Quality Entrance and Exit Criteria for SLT • Ensures service usability

21. Security Testing and the Project Lifecycle • Security requirements should be established • Security risk assessments should be performed during the technical design phases • Technical deliverables should be validated in accordance with the groups security standards • Security tests should be performed at the service level and not just the delivered integrated system level • Security audits should be performed and reported on periodically

22. SOA and NAS Experts Leverage • Subject Matter Experts / Domain Experts • NAS expertise • SOA domain expertise • Software development expertise • Certified Resources • Certified individuals • Certified tools • Software Support • Commercial Off-The-Shelf (COTS) support contracts • Open Source community and committers • Knowledge Sharing • Content repositories (SWIM Wiki, Knowledge Services Network (KSN)) • Working groups

23. Test Tool Strategy Implementation • SOA Test Tools • iTKO LISA, soapUI • XML Tools • Altova XMLSpy, XML Copy Editor • Integrated Development Environments (IDEs) • FUSE IDE for Camel, Eclipse IDE, NetBeans IDE, IntelliJ IDEA • Web Consoles and Browsers • Web consoles for ActiveMQ, Camel, Oracle AS, Glassfish, etc. • Fuse HQ, Lightweight Directory Access Protocol (LDAP) browsers, management and monitoring tools

24. Return On Investment

25. Business and Financial Metrics • Efficiencies associated with service reuse • Integration time savings • Related opportunity costs • Cost savings/avoidance • Reduction in project and maintenance cost SOA ROI ($) = Cost Savings/Efficiencies Achieved - All SOA-Related Investments SOA ROI (%) = SOA ROI ($) All SOA-Related Investments Source: Leo Shuster, who is responsible for National City Bank's SOA

26. SOA Performance Metrics • Overall ROI can be projected by per service ROI’s • Overall ROI can be projected by per service revenue • Monitor service growth rate/reuse to ensure reuse maximization • Design to deploy business agility • Mean time to service change agility measurement • Reliability mean time to failure • Service vitality and revenue tracked for 12 months

27. Anticipated Future Challenges • Services Metadata • Employing Governance in regards to managing and providing information on how services interact • Level of Security • Protection against theft of sensitive data and application infrastructure vulnerabilities • Test Coverage • Continuous services testing by producers and consumers to ensure expected functionality • Scalability • Specialized support for persistence, failover, and load balancing • XML Message Size • Additional bandwidth and resources for parsing/processing • Increase in Connections • Connection management for server burden relief • Shared services hosting to avoid latency from additional hops

28. Conclusion • Traditional Testing has its place • Migrate to SOA where appropriate • SOA brings benefits such as loose coupling, code/services reuse, and remote users • SOA test products are numerous, mature and vary for many different use cases • SOA challenges have been identified with many modeled solutions available to pursue

29. Questions and Comments

30. Back up

31. Keys to a Successful SOA Solution Definition of SOA Find Business Partners Aim for Critical Mass Franchise