standards conformance testing and ms web services

1. Standards Conformance Testing and M&S Web Services Bernard P. Zeigler, Ph.D., Arizona Center for Integrative Modeling and Simulation and Joint Interoperability Test Command Fort Huachuca, AZ 85613-7051

3. Net-Centric Test and Evaluation: Some Assertions Net-Centric T&E places an increased emphasis on standards conformance testing (SCT) SCT will be the first gate before more extensive T&E necessary, but not sufficient, for interoperability and mission effectiveness performed on distributed net-centric test-bed infrastructure Global Information Grid (GIG) provide on-site or downloadable SCT packages emphasizes services rather than systems testing of Service Oriented Architecture (SOA) GIG enterprise services

4. JITC is the Responsible Test Organization for Tactical Data Link Standards Joint Interoperability Test Command (JITC) is responsible for ensuring systems that implement Tactical Data Link* (TDL) are interoperable and in compliance with the applicable joint standards This is accomplished by conducting the following types of tests: Joint / NATO /Combined Interoperability Performance Assessment in Operational Environments Standards Validation Standards Conformance JITC employs a variety of tools that provide its analysts the ability to evaluate TDL system performance in both the lab and live environments.

5. Link-16: Challenges to Implementation and Testing Joint Single Link Implementation Requirements Specification JSLIRS is an evolving standard (MIL-STD-6016c) for tactical data link information exchange and networked command/control of radar systems Presents significant challenges to automated conformance testing: The specification document states requirements in natural language open to ambiguous interpretations The document is voluminous many interdependent chapters and appendixes labor intensive and prone to error potentially incomplete and inconsistent. Problem: how to ensure that a certification test procedure is traceable back to specification completely covers the requirements can be consistently replicated across the numerous contexts military service, inter-national, and commercial companies

6. Benefits of Formalization and Automation Provides traceability to original specification Reduces ambiguity from textual specification Facilitates integrating Modeling & Simulation into the testing process Enables testing of complex: Standards Systems Functions Families of systems

7. SIAP/IABM —Successor to Link-16 SIAP (Single Integrated Air Picture) Goal: Improve the adequacy and fidelity of information to form a shared understanding of tactical situation Integrated Architecture Behavior Model (IABM) requires that all sensors utilize a standard reference frame for conveying information about the location of targets. Developed by the Joint SIAP System Engineering Organization (JSSEO), Arlington, Va., a sub-office of the Assistant Secretary of the Army for Acquisition, Logistics and Technology.

8. Automated Test Case Generator (ATC-Gen)

9. Discrete Event Nature of Link-16 Specification

10. ATC Gen Goals & Approach

11. ATC Gen Tool Standard to XML Translation Analyst interprets the requirements text to extract state variables and rules, where rules are written in the form: If P is true now Condition Then do action A later Consequence Unless Q occurs in the interim Exception Dependency Analysis & Test Generation Dependency Analyzer (DA) determines the relationship between rules by identifying shared state variables Test Model Generator converts Analyst defined test sequences to executable simulation models Test Driver Test Driver interacts with and connects to SUT via HLA or Simple J interfaces to perform conformance testing Validated against legacy test tools

12. Capturing the Link-16 Standard in XML

13. Rule AnalysisExtract I/O Behaviors

14. Test SequenceManually Derive & Create Paths

15. Test SequenceValidation & Generation

16. Test CaseGeneration

17. Test ModelValidation & Generation

18. Test ModelExecution

19. Extending M&S-Based T&E to the GIG Migrate ATC-Gen capability to the GIG-SOA accessible to all developers requiring Link-16 conformance testing and certification enable self-tests for Link-16 standards conformance expose Link-16 XML-rule repository as authorized reference model register ATC-Gen test capabilities as web services Extend M&S-Based automated testing provide mission thread testing for GIG deployable services certify transitions through the stages of DISA’s Federated Distributed Certification Environment Expose M&S-based T&E assets as services experimental frame components: generators, acceptors, transducers exploit DEVS reusability support characterize services for discovery and reuse

20. Link-16 XML Repository Web-Service Asset

21. Transferring DEVS-based Testing Methodology to SOA

22. Automating Mission Thread/Vignettes for GES Testing Objective – automate to the maximum extent the definition of mission threads/vignettes and the generation of test services based on them Enable COI and community wide service users and developers to easily write threads/scenarios in (restricted) Natural Language form Interacting with such users/developers, implement these scenarios with DEVS test federation models that can execute over Core ES Mission thread/Vignette scenario testing applies objectives-driven testing to all service levels and throughout service offering life cycle Repository can support re-use and re-application for regression testing to preserve service behavior under continual change in SOA environment

23. Mission Thread Testing Applies to all Service Levels and Throughout Service Offering Life Cycle

24. NLP Mission Thread Scenario Generation

25. Example: Joint Close Air Support Scenario

26. NLP Test Generation Example: Joint Close Air Support Scenario

27. DEVS-based Web-Services Testing

28. Demo:http://acims9.acims.arizona.edu

29. M&S-Based T&E Services on the SOA:Experimental Frame Repository Compose EF from generator, acceptor, and transducer components in repository Characterize functionalities using M&S Theory and WSDL Support composability based on migration of DEVS technology to SOA

30. Example: Synthetic Environment Repository

31. Summary and Conclusions There is an acute need for a new Net-centric testing paradigm at the enterprise level where joint and coalition operations are conducted (Editorial, ITEA Jnl, Sept, 2005) Net-Centric T&E places an increased emphasis on standards conformance self-testing over the GIG-SOA and emphasizes services rather than systems Due to its formal and abstract basis, the DEVS functionality migrates quite directly to execute over the web-services middleware based on XML and SOAP This extension enables semi-automated construction of DEVS test federations for verification testing of a wide variety of proposed web services: ATC-Gen Link-16 and other TDL standards conformance self testing Mission thread or vignette scenario specification using natural language input can replace Link-16 style standards formalization as the source input for test case generation M&S-based T&E services can exploit experimental frame methodology