1 / 23

WS-VLAM Introduction

Systems and Network Engineering group Institute of informatics University of Amsterdam. WS-VLAM Introduction. Introduction. This presentation gives a quick introduction to the WS-VLAM workflow management system

cotis
Download Presentation

WS-VLAM Introduction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WS-VLAM Introduction presentation Systems and Network Engineering group Institute of informatics University of Amsterdam WS-VLAM Introduction

  2. WS-VLAM Introduction presentation Introduction • This presentation gives a quick introduction to the WS-VLAM workflow management system • It helps the reader understand the technology push which motivated our design, describes the current state and present the ongoing activities and planned ones. • NOTE: • We are currently still working on the User’s and developer’s Guide, they are notavailable at this time. The presentations do not replace the documentation it just give a quick introduction. • Do not hesitate to contact us for further information Thanks for taking some of your time to read about the WS-VLAM work, any comments are more than welcome WS-VLAM developers gvlam-dev@lists.vl-e.nl

  3. WS-VLAM Introduction presentation Needed background … • To be able to follow the content of this presentation you will need to know about: • WSRF= Web Service Reference Framework • GT4= Globus Toolkit version 4 • More detailed description of the presented work can be found in the section of publication of the WS-VLAM home page

  4. WS-VLAM Introduction presentation Objective of this presentation… • Why you should look at this presentations • You want to know how WS-VLAM works and interacts with the Grid middleware. • Why you don’t have to worry if you do not understand the content on this presentation: • You are an end-user: it will be enough for you to read the User’s guide or the WS-VLAM composer presentation • You are an application developer: it will be enough for you to read the Developer’s guide & the WS-VLAM developers presentation

  5. WS-VLAM Introduction presentation Outline • Introduction • Architecture of the WS-VLAM • Hierarchical workflow compositions & execution • Interoperability with other WMS • Conclusions

  6. WS-VLAM Introduction presentation GVLAM: • Enable end-users (scientists) todefine, execute, and monitor e-science applications. • Location independent experimentation. • Application Domain portal. • Provide application developers: • Abstract view of the Grid (shields the Grid details). • Easy way to port applications to the Grid.

  7. WS-VLAM Introduction presentation The Virtual Laboratory for e-Science … Food Informatics Dutch telescience Medicaldiagnosis Bio diversity Data intensivescience Bioinformatics ASP Application layer Generic e-science framework layer Grid layer

  8. WS-VLAM Introduction presentation Before and after OGSA/WSRF User Applications Application Templates Workflow Management system Data Management Stack Process Management Stack Web Service Interface Web Service Interface Grid Middleware: Process & resource management Grid Middleware: Data management Network & storage Resources Network & Computing Resources

  9. WS-VLAM Introduction presentation Workflow Management Systems User Applications Application Templates Workflow composition Workflow Management system Workflow execution Data Management Stack Process Management Stack Web Service Interface Web Service Interface Grid Middleware: Process & resource management Grid Middleware: Data management Network & storage Resources Network & Computing Resources

  10. WS-VLAM Introduction presentation Grid Service & OGSA-DAI Grid Services Grid Services Grid Services User Applications Application Users Application Templates Workflow composition Workflow Management system Workflow execution Data Management Stack Process Management Stack Web Service Interface Web Service Interface OGSA DAI Grid Middleware: Process & resource management Grid Middleware: Data management Network & storage Resources Network & Computing Resources

  11. WS-VLAM Introduction presentation First Application developers … User Applications Application Templates Workflow composition Workflow Management system Workflow execution Data Management Stack Process Management Stack Web Service Interface Web Service Interface Grid Services Application Templates Web service OGSA DAI Application web service Grid Services Grid Services Grid Middleware: Process & resource management Grid Middleware: Data management Network & storage Resources Network & Computing Resources

  12. WS-VLAM Introduction presentation Second the Workflow developers … Workflow composition Workflow Management system Distributed Workflow Workflow execution Data Management Stack Process Management Stack Web Service Interface Web Service Interface Grid Services Workflow Engine Web service Application Templates Web service OGSA DAI Application web service Grid Services Grid Services Workflow Engine Web service Grid Middleware: Process & resource management Grid Middleware: Data management Network & storage Resources Network & Computing Resources

  13. WS-VLAM Introduction presentation WS-VLAM WMS… • Goals • Create a WMS which adopt a Service Oriented Paradigm • Approach • Workflow Engine (RTSM) • Implemented as a WSRF service using Globus 4 (GT4) • Workflow Composer (GUI) • Thin client • Repository for workflow components • Implemented also as WSRF service.

  14. WS-VLAM Introduction presentation Conclusions • WSRF compliant middleware such as GT4 offers useful basic services to develop a truly grid enabled workflow engine: • Delegation service and ws-notification mechanisms proved to be very useful • Separating the workflow composition from execution suits better the Grid environments and improve interoperability • OGSA-WG is collecting cases and input to decide which services should be standardized

  15. WS-VLAM Introduction presentation http://www.vl-e.nl/

  16. WS-VLAM Introduction presentation New workflow Components • The workflow components developed the last year, can be still used and will be deployed on the shared workflow component repository on the rapid prototyping environment • Ridge-O-Grammar modules from (SP 1.5) • Kansk Toolbox from (SP 1.4) • AID web services (SP 2.2) • Virtual file system Web service (SP 2.4) • Generic workflow components (SP 2.5) • File Reader/Writer, • universal File Reader/Writer (SRB/GFTP/...) • Demos: MRIscanner, Histogram, Floatingball • We will contact other SPs to integrate their components • Legacy code, or fully integrated code (java, C++, Python)

  17. WS-VLAM Introduction presentation other tasks • Access to the ws-vlam engine from Taverna • Access to the ws-vlam engine from Kepler • Parameter sharing among workflow components • Job farming • Support for more than one job submission mechanisms (GT5-GRAM, Direct, etc) • Fault tolerance • Control flow • Data provenance • Semantic Service discovery

  18. WS-VLAM Introduction presentation References and download • VLAM client can be downloaded from • www.science.uva.nl/~gvlam/wsvlam/ • Documentation can be downloaded www.science.uva.nl/~gvlam/wsvlam/doc/

  19. WS-VLAM Introduction presentation Workflow interoperability: Execute a VLAM workflow from Kepler/Taverna • A predefined Application workflow developed in VLAM can be executed as a single step in Kepler/Taverna • (no need to recompose graphically the whole workflow). • The predefined Application workflow will be executed on any remote computing resource where the VLAM-RTSM GT4 Web service is installed. • Advantages: • Compose workflow where sub-workflows (which require grid resources) are executed on grid-enabled resources, while the rest of the workflow is either executed using other Kepler actors or taverna processors • It is also more efficient, since it avoid the overhead which will result by wrapping every workflow component as a separated web service or a separate remote grid-execution.

  20. WS-VLAM Introduction presentation Workflow interoperability: Execute a VLAM workflow from Kepler/Taverna • A predefined Application workflow developed in VLAM can be executed as a single step in Kepler/Taverna • (no need to recompose graphically the whole workflow). • The predefined Application workflow will be executed on any remote computing resource where the VLAM-RTSM GT4 Web service is installed. • Advantages: • Compose workflow where sub-workflows (which require grid resources) are executed on grid-enabled resources, while the rest of the workflow is either executed using other Kepler actors or taverna processors • It is also more efficient, since it avoid the overhead which will result by wrapping every workflow component as a separated web service or a separate remote grid-execution.

  21. WS-VLAM Introduction presentation Workflow interoperability: Execute a VLAM workflow from Kepler/Taverna Kepler/Taverna workbench RTSM-GT4 Web service Das2 or PoC facilities. GT4 Java Container GRAM services (2) Service Invocation RTSM Factory pre-ws-GRAM VLAM Actor or Taverna processor RTSM Client RTSM Instance Worker nodes Workfow Description (XML) (1) Proxy Delegate Delegation service Workflow components • Kepler/Taverna users can have access to some of the parameters of the Application workflow to change the default values • Kepler/Taverna users have to specify the location of the input data file as URL and will get back a URL if the Sigwin generate data files • Graphical output of the Application workfloware handled automatically by the VLAM Taverna processor /Kepler actor.

  22. WS-VLAM Introduction presentation Some References • Science Z Zhao, A Belloum, M Bubak Editorial: Special section on workflow systems and applications in e-Future Generation Computer Systems 25 (5), 525-527 • A. Wibisono, D. Vasyunin, V. Korkhov, AS.Z. Belloum WS-VLAM: a GT4 based workflow management system , The 2nd International Workshop on Scientific Workflows, In conjunction with ICCS 2007, Beijing, China, 2007 • A.S.Z. Belloum, V. Korkhov, S. Koulouzis, M. A Inda, and M. BubakCollaborative e-Science experiments: from scientific workflow to knowledge sharing JULY/AUGUST, IEEE Internet Computing, 2011 • IlkayAltintas, Manish Kumar Anand, Daniel Crawl, Shawn Bowers, Adam Belloum, Paolo Missier, Bertram Ludascher, Carole A. Goble, Peter M.A. Sloot, Understanding Collaborative Studies Through Interoperable Workflow Provenance, IPAW2010, Troy, NY, USA

  23. WS-VLAM Introduction presentation Some References • A. Belloum, Z. Zhao, and M. Bubak Workflow systems and applications , Future Generation Comp. Syst. 25 (5): 525-527 (2009) •  Z. Zhao, A.S.Z. Belloum, et al., Distributed execution of aggregated multi domain workflows using an agent framework The 1st IEEE International Workshop on Scientific Workflows, Salt Lake City, U.SA, 2007 • Zhiming Zhao, Adam Belloum, Cees De Laat, Pieter Adriaans, Bob Hertzberger Using Jade agent framework to prototype an e-Science workflow bus Authors Cluster Computing and the Grid, 2007. CCGRID 2007

More Related