1 / 21

DWD GISC Node Current Status

DWD GISC Node Current Status. Agenda. Welcome Current status: Overview based on functional blocks High Level Architecture Web Interface DAR Component Catalogue systems, Metadata and identified issues GISC-to-GISC communication Experience with Blog solution and AFD

hedwig
Download Presentation

DWD GISC Node Current Status

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. DWD GISC NodeCurrent Status

  2. Agenda • Welcome • Current status: Overview based on functional blocks • High Level Architecture • Web Interface • DAR Component • Catalogue systems, Metadata and identified issues • GISC-to-GISC communication • Experience with Blog solution and AFD • Schema for GISC backup scenario • Data Components • MSS, AFD and subscription management • Live Demo • Challenges during the Implementation • Q&A

  3. High level architecture: Functional blocks Logging, Monitoring and Reporting Authorisation/Authentication services SubscriptionManagement Gray List Container WMO GISC-GISC Interface • Delivery tools • Moving Weather • AFD • controller • Datapool(s) • controller/manager • access adapter • data sources Input Management Receiver External DCPC • Web functions • Frontend • Web services (interfaces) • Web services (core) • Business logic • DAR component • controller/manager • catalogue • editor DAR-Generator/Importer

  4. Web functions • Frontend • Web services (interfaces) • Web services (core) • Business logic • DAR component • controller/manager • catalogue • editor HLA: Web Interface (User/Admin/Customer) Logging, Monitoring and Reporting Authorisation/Authentication services SubscriptionManagement Gray List Container WMO GISC-GISC Interface • Delivery tools • Moving Weather • AFD • controller • Datapool(s) • controller/manager • access adapter • data sources Input Management Receiver External DCPC DAR-Generator/Importer

  5. HLA: Web Interface (User/Admin/Customer) • WebWerdis • Redesign of WebWerdis • Navigation structure / search functions • Internationalization • Integrate logging framework (necessary for monitoring) • Code clean-up • Integration with Geonetwork • Geonetwork • Customization of GUI

  6. HLA: Web Interface (User/Admin/Customer) • Search by URL – ISO 23950 • Timo (WMO Secretariat) did a SRU implementation in Geonetwork • Solution is running in DWD DMZ • 3 day implementation workshop with Timo about SRU and jzkit at DWD • Direct integration into DWD GISC node

  7. SubscriptionManagement Gray List Container • Datapool(s) • controller/manager • access adapter • data sources Receiver • DAR component • controller/manager • catalogue • editor DAR-Generator/Importer High level architecture: Metadata Management Logging, Monitoring and Reporting Authorisation/Authentication services WMO GISC-GISC Interface • Delivery tools • Moving Weather • AFD • controller Input Management External DCPC • Web functions • Frontend • Web services (interfaces) • Web services (core) • Business logic

  8. High level architecture: Metadata Management • DAR Catalogue • Did extensively Performance Tests (Geonetwork 2.2, terraCatalog 2.3) • DWD formulated test cases • Contracted external partner for tests • Functional Tests • DAR Generator / Importer • Developed DAR metadata generator (template driven architecture) • External review of DAR metadata • Developed DAR importer • Issues we identified during our work • Tested DAR catalogues fulfilled most, but not all INSPIRE and WMO requirements • DAR Metadata • Paper with Request for Clarification (JMA, CMA and DWD) • Schematron rules for validating WMO profile would be very helpful

  9. High level architecture: Metadata Management • Metadata Harvesting • CMA and DWD did several functional tests with OAI-PMH in 09/09 • Different implementations of OAI-PMH could not harvest each other • Deleted metadata records: different implementations • Standard vanilla implementations modify harvested metadata • Ongoing tests between CMA and DWD • Started January 2010 • Initial tests showed 1 record ~ 1 second  space for improvement • Backup strategy • Exchange of metadata files as products

  10. High level architecture: Metadata Management

  11. High level architecture: G2G communication Logging, Monitoring and Reporting Authorisation/Authentication services SubscriptionManagement Gray List Container WMO GISC-GISC Interface • Delivery tools • Moving Weather • AFD • controller • Datapool(s) • controller/manager • access adapter • data sources Input Management Receiver External DCPC • Web functions • Frontend • Web services (interfaces) • Web services (core) • Business logic • DAR component • controller/manager • catalogue • editor DAR-Generator/Importer

  12. HLA: GISC-to-GISC communication • Collaboration CMA and DWD • Memorandum of Understanding signed end of last year • Two Workshops held • GISC-to-GISC communication tests with CMA • CMA and DWD installed Blog solution and AFD solution • Data sets with appropriated time profile (issue warning during model transfer) • Blog solution • CMA and DWD did several improvements on publishing and polling components • Usage of tar minimize latency of transfer • Necessary to include size of “metadata” (AtomFeed) into calculation of transfer volume • Identified several further improvements like prioritization, AtomFeed handling, logging • JMA has provided a new improved solution in 10/2009 • AFD solution • Average transport time between 4-5 sec during 24hour continuously test, warnings between 3-4 sec • In case of unsecured connection sometimes corrupted data occur

  13. HLA: GISC-to-GISC communication backup • Technical specification requires a bilateral agreement from 2 GISC for a backup scenario • CMA and DWD prepared proposal for priorities in DAR metadata • First common draft of dissemination metadata (XML schema) • Need to prove in practice

  14. SubscriptionManagement • Delivery tools • Moving Weather • AFD • controller • Datapool(s) • controller/manager • access adapter • data sources High level architecture: Delivery components Logging, Monitoring and Reporting Authorisation/Authentication services Gray List Container WMO GISC-GISC Interface Input Management Receiver External DCPC • Web functions • Frontend • Web services (interfaces) • Web services (core) • Business logic • DAR component • controller/manager • catalogue • editor DAR-Generator/Importer

  15. High level architecture: Delivery components • Message Switching System • REST Web-Service to manage and control Moving Weather • First proposal for generic Interface • Integration into WebWerdis (still under work) • Subscription Management • Controller: time and event triggered management (still under work) • Customization according to GISC-to-GISC backup necessary

  16. Live Demonstration

  17. TC 5 - Maintenance of Users, Roles, Authorization and Authentication • Search metadata • Access data • Show difference between authorized and unauthorized User • User and Customer management • Subscription • View historic transactions

  18. Metadata Management – TC 1 • Show metadata editor • Fully integration open because of separation between Intranet and DMZ • Authorization required to add/modify/delete metadata

  19. Metadata Management – TC 2, TC 6 • A WIS Centre provides new metadata records • Show that GISC Geonetwork doesn’t contains new metadata records • Show that GISC_PRODUCTS is empty • Copy new metadata records to remote provider, reindex • GISC harvests new metadata records • Update of GISC Geonetwork and jOAI Provider • Show update of Geonetwork with SRU query • Show update of GISC_PRODUCTS

  20. Challenges during the Implementation • Metadata generation • See paper “Questions on Metadata” from CMA, JMA and DWD from 11.2009 • WIS Specification is open – what is good!

  21. Q&A

More Related