Virtual Observatory and Data System Interoperability

1. Virtual Observatory and Data System Interoperability Michele Weiss, Daniel Morrison, Rose Daley, Elisabeth Immer, Mo Hashemian, Robert Holder, Julia Jen, Brand Fortner and Eric Hu The Johns Hopkins University Applied Physics Laboratory michele.weiss@jhuapl.edu 240-228-4806

2. Current Virtual Observatories Virtual Observatories being implemented on a domain specific basis As VOs evolve, interoperability between VOs will become crucial To enable multi-mission and multi-discipline scientific research Need to provide access to heterogeneous resources including both new and legacy data systems as well as to other VOs

3. What Does Access to Heterogeneous Resources Mean? • A unified system for all data systems and science disciplines is impractical • Common data model will never be a perfect fit for all systems • “One Size Fits All” User Interface can’t incorporate full functionality of distributed data system features • Taking advantage of portals that connect to other VOs and other heritage systems is much more viable • Portals, however fail in the integration of resources • Data model mapping just a small subset of making systems interoperable

4. The Current Environment • VOs teams are talking to each other • SPASE is defining an extensible data model • Currently only describes datasets • But…. • How will extended data models “talk” or map to each other? • What about other types of metadata needed to discover and use the data/system? • What about other resources that data systems and VOs provide such as tools, services, models and User Interfaces?

5. Our Solution • Use loose coupling and map commonly understood scientific concepts rather than exact metadata matchups • Ongoing research in Semantic Web technology (e.g., ontologies) could facilitate this • Model eliminates the need for rigid metadata standards or unique, resource-specific integration software • Use standard methods like web services to integrate services such as query interfaces and services into our system • Increase functionality by effectively joining systems into a larger virtual collective • Provide access not just to data models but also tools, services and models

6. Accessing Data Only Part of It Interoperability is more than just getting data Most common service offered is “Get Data” Systems can be viewed as collections of available “services” Most systems have additional services (e.g., format conversion) Often not externally available as services Some of these services may be interactive SRAS SPDML

7. VO Service Based Overview • Most systems have additional services (e.g., format conversion) • There are different types of services • Some of these services may be interactive

8. Accessing Services • Services provide significant new capabilities • “Standard” ability to retrieve data • Utilities to post-process data (e.g., format conversion) • Run programs (e.g., data assimilation models) • Improve data queries (e.g., coincidence calculation or provide better query interface) • Integrating these capabilities can be complicated • Sequence of services may be needed

9. Using Services Effectively Create a query • Services can be utilized to improve and refine data retrieval • Services can be used to perform operations beyond data retrieval • Systems using a service need to: • Understand how to use a service • Understand when to use the service • Not require a new software release to be able to use the service • Visualize data (e.g., stack plots of selected time series data) • Generate data summaries (e.g., min-max-average values) Refine a query Search metadata for matches Review possibilities • Calculate spatial or temporal parameters • Visualize spacecraft orbits • Visualize instrument observed region • Use a different interface to improve query Select Resources Retrieve Resources • Convert format • Automatically send data to another service (e.g., data assimilation model, data mining service)

10. Front Door Integration • We have prototyped and developed a new approach that can allow many of the capabilities of an existing data center to be easily incorporated into the SRAS by interfacing to their system by a specialized web service • Consider, for example, a data center that provides data query and retrieval services. A significant amount of time and effort will certainly have been spent developing the user interface to such a system • We have developed services that encapsulate that entire user interface, allowing user input, enabling a combined system to take full advantage of the rich capabilities that already exist • We demonstrated the feasibility of this approach by incorporating the Space Physics Data Markup Language search engine (SPDML) into the SRAS system • Advantages • Allows reuse of user interfaces • Concept mapping is high level (user level) rather than at the database or metadata level – much simpler. • Allows integration of extensive search services. SPDML SRAS

11. Demonstration

12. Demonstration – cont. • Enhanced search services • Internal and external services and tools • Demonstration uses web services • Other technologies can be easily incorporated in future

13. Demonstration – cont.

14. Demonstration – cont.

15. Demonstration – cont.

16. Demonstration – cont.

17. Invoking Any Interoperable Service • We call this “Front Door Integration” • As long as we know: • Where service or tool is located • Can “talk” to service using industry stands for content and format, i.e. SQL instead of custom API • Can generate 2-way translation mapping between SRAS and service

18. Front Door Integration • Provides enhanced search services that go beyond a single data center • Allows many of the existing as well as new capabilities of data systems and centers to be easily incorporated • Can include services that encapsulate the entire user interface, allowing user input enabling a combined system to take full advantage of other rich capabilities

19. Example Services - SPIDR

20. Example Services – CDAWeb+

21. Example Services – TIMED Coincidence Calculator

22. Summary • Front Door Integration easily integrates disparate collections of resources including interfaces through a single portal • Without resorting to “one size fits all” • Enables discipline specific interfaces tailored for scientists • True interoperability goes beyond data integration • Services provide the maximum capability • Minimizes duplication of effort; enables reuse of “already invented wheels” • Innovative technology like SRAS, which will be incorporated into VITMO, makes it easy to incorporate resources from existing VOs and data systems (not just new ones) • Legacy systems can be cost effectively integrated into new technology based systems w/out requiring redesign • Can tie in existing systems (like NGDC) as well as introducing new capabilities