Richard Hull (Bell Labs) Jianwen Su (UC Santa Barbara)

1. Tools for Design ofComposite Web Services -- Presented Version (June 17, 2004) -- http://www.cs.ucsb.edu/~su/tutorials/sigmod2004.html Richard Hull (Bell Labs)Jianwen Su (UC Santa Barbara)

2. Outline • Introduction and positioning • Standards • Models of Web Services and Composition • Approaches to Automated Composition • Analysis and Verification • Future Directions SIGMOD'04

3. Web Services • The Web: Flexible human-machine interaction • Web Services: Flexible machine-machine interaction • Working Definition: Network-resident software services accessible via standardized protocols • Simple Object Access Protocol (SOAP): very flexible remote procedure call • Lots of interest in trade press, academic community, standards bodies, . . . • Applications in e-commerce, telecom, science, GRID, government, education, . . . SIGMOD'04

4. Web Services: The Big Questions Primary focus of this tutorial Simplify and/or automate web service • Discovery • What properties should be described? • How to efficiently query against them? • Composition • Specifying goals of a composition • Specifying constraints on a composition • Building a composition • Analysis of compositions • Invocation • Keeping enactments separated • Providing transactional guarantees • Monitoring • How to track enactments • Recovering from failed enactments SIGMOD'04

5. Anatomy of Web Services Composition • No unified model, e.g., • BPEL: Strong on orchestration, info sharing • OWL-S: Strong on goals, activities, discovery • “Roman” model: Strong on activities, orchestration Goal(s) Activities Orchestration, Monitoring Discovery/ Self-description Info sharing (Messaging) SIGMOD'04

6. Key dimensions in web service composition BPEL OWL-S “semantics” Commitment Protocols CTR-S -Calc WSCL Mealy BPML Complexity of glue language Roman CSP WSDL Complexity of component services SIGMOD'04

7. Key Disciplines for Web Services Composition • Science of Design, e.g., • Workflow • Service-oriented computing • Reactive systems • Process algebras • Verification • AI, e.g., • Knowledge Representation • Agents • Planning • Database, e.g., • XML • Domain-specific query languages • Indexing • Transaction Management SIGMOD'04

8. Goals of this Tutorial: Tools for Composition • Describe the playing field • Key standards that we’ll all build upon • Promising models and formalisms • Results on Composition • OWL-S community • Automata-theoretic approaches • Results on Analysis and Verification • Workflow community • OWL-S community • Automata-theoretic • Future directions SIGMOD'04

9. Outline • Introduction and positioning • Standards • Models of Web Services and Composition • Approaches to Composition • Analysis and Verification • Future Directions SIGMOD'04

10. Web Services Standards Stack: Key Elements UDDI Discovery WS-Choreorgaphy Choreography BPEL4WS OWL-S ServiceModel Composition WSCL (Individual)ServiceDescription WSDL OWL-S ServiceProfice XMLMessaging SOAP HTTP, SMTP, FTP, etc. Network SIGMOD'04

11. SimpleObjectAccessProtocol • A W3C standard • Originally developed for BizTalk • A light weight replacement for complicated distributed object technology • “XML-RPC”, typically through HTTP, also JMS … • Lowest level of service interaction Web Server SOAP Envelope External Service Web Service SIGMOD'04

12. RPC Messages • Typically two messages Request Message SOAPClient SOAPServer Response Message SOAP Envelope “RPC style” SOAP body encodes • the operation name and parameters • return result XMLified SOAP Header SOAP Body SIGMOD'04

13. Web Service Definition Language (WSDL) • WSDL provides a framework for defining • Interface: operations and input/output • Access specification: SOAP bindings (e.g., RPC) • Endpoint: the location of service Supports Port Type Operation How to invoke Formats & Protocols Input & Output How to encode Binding Message Implements Provides Port Service [from Leymann BTW 2003 talk] SIGMOD'04

14. WSDL Operations • Traditional I/O signatures (using XML Schema) • Four operation types • Proactive : send request send request, block till response • Reactive : receive request receive request, send response bill_payment out: bill in: payment order bill order Supplier’ Supplier receipt payment receipt • Port: mechanism to cluster operations • Port as unit of interoperation between services SIGMOD'04

15. Business Process Execution Language (BPEL) • Allow specification of compositions of Web services • business processes as coordinated interactions of Web services • Allow abstract and executable processes • Influences from • Traditional flow models • Structured programming • Successor of WSFL and XLANG • Assumes WSDL ports • Standardization through OASIS SIGMOD'04

16. BPEL in Action operations Purchase Order service coordinates other services using ports in WSDL Purchase Order portType Receive Purchase Order Initiate Price Calculation Decide On Shipper Initiate Production Scheduling Price Calculation portType Arrange Logistics Complete Price Calculation Complete Production Scheduling Invoice Processing [from BPEL 1.1 standard] SIGMOD'04

17. BPEL Activities • Invokes an operation on a partner service • Send to WSDL port, wait for a response • Receives invocation from a partner • Wait for a message • Sends a reply message in partner invocation • Send a message (corresponding to some earlier message) • Data assignment between containers • Copy local data • Control structures: sequence, flow (possibly with links), pick, loops, etc. • Scoping, exceptions, compensation SIGMOD'04

18. BPEL Examples begin parallel • Flowcharts “with parallelism” • “Pick” construct to enable waiting for input (or time out) • Synchronization within parallel threads • Comparison of supported constructs: see [van der Aalst ’03] Initialize do until flag Receive Bill1 send Order end_date reached pick receive order Send Bill receive Receipt1 case flag := true Receive Payment suppl1 order suppl2 order send Receipt Send Payment1 end case end parallel SIGMOD'04

19. Web Service Conversation Language (WSCL) Purchase ?PurchaseOrder !POAccepted Shipping !ShippingInfo !OutOfStock !InvalidPayment • A key to web service composition: • Interactions between services • WSCL specifies a conversation (behavior signature) as a labeled graph: • Nodes: interactions, individual units of responses • Edges: transitions, sequencing of interactions • Edge labels: conditionson transitions InvalidPayment POAccepted SIGMOD'04

20. WS Choreography • An emerging standard from W3C • Drawing inspiration from the p-calculus • Global view of composite service interactions • Global model: interactions and choreography • Choreography Definition Language (WS-CDL) • Key technical elements • Participants and roles: what services are involved • Channels: where and how the messages are sent • Interactions: message exchange patterns • Activities and control structures: sequencing • Choreography: a global description of a composition • Interactions, exceptions, finalizer • composable SIGMOD'04

21. A scripted composition using WSDL messages Control structures with constrained parallelism More procedural Executable or abstract Favor centralized composition A global description of what and how WDSL messages are exchanged Declarative flavor Abstract and not executable (yet) Composition infrastructure neutral Channels can be passed around (e.g. p-calculus) BPEL meets WS-Choreography SIGMOD'04

22. OWL-S (Formerly DAML-S) • An emerging standard to add semantics: • An upper ontology for describing properties & capabilities of web services using OWL • Enable automation: service discovery & selection, invocation, composition & interoperation, executionmonitoring • input types • output types • preconditions • effects Resource ServiceProfile provides presents (what it does) Service describedby(how it works) supports(how to access) ServiceGrounding ServiceModel • communication protocol (RPC, HTTP, …) • port number • marshalling/serialization • process flow • composition hierarchy • process definitions SIGMOD'04

23. OWL-S Service Profiles • Input types • Output types • Preconditions • Effects ServiceProfile presents (what it does) • Service profile defines what the service provides: • Functional descriptions: In/Output, Preconditions, Effects • Non functional descriptions: name, category, QoS, … • Can use situation calculi (e.g. PSL) as formal basis for pre-conditions, effects: • Assume a world of “fluents” – typically a set of propositions, where actions make some true, some false • Reasoning with pre-conditions and effects • Service profiles are hierarchically organized (example later) Service SIGMOD'04

24. OWL-S Service Model SIGMOD'04

25. OWL-S Process Model Service describedby(how it works) • Constructs for composite processes • Sequence • Concurrency: Split; Split+Join; Unordered • Choice • If-Then-Else • Looping: Repeat-Until; Iterate (non-deterministic) • Note: In spirit of Golog, these can be viewed as constraints • Data Flow • No explicit variables, no internal data store • Predicate “sameValues” to match input of composite service and input of subordinate service • Less refined than, e.g., BPEL • Message behavior of composed OWL-S services not well-understood ServiceModel • process flow • composition hierarchy • process definitions SIGMOD'04

26. Universal Description, Discovery and Integration (UDDI) • Directory for web services • Communicate via SOAP • Includes descriptions of services, in terms of: • Business, services, binding, “technical fingerprints” • tModels • “Schemas” for describing service templates (PortTypes) • There are tModel’s for WSDL descriptions of a service, for ebXML, … • When a service registers with UDDI, the technical fingerprint includes listing of tModels that it uses • tModel’s can be registered, and incorporated into taxonomies • Allows queries over services, tModels, implementations, and other information • UDDI expected to expand over time, enabling richer service descriptions SIGMOD'04

27. OWL-S Profile Ontology is Analogous to the Concept of UDDI Taxonomy SIGMOD'04

28. Outline • Introduction and positioning • Standards • Models of Web Services and Composition • Approaches to Automated Composition • Analysis and Verification • Future Directions SIGMOD'04

29. Models of Interoperation Different models focus on different aspects • Automata-based • Intricate structure for atomic services • Rich interleaving between atomic services • Message-based or activity-based • Logic-based perspectives • Frameworks supporting proof and model theories • Different ways of modeling complex services • Natural to incorporate “effect on the world” • Constraint-based • Support partial specification of desired behaviors • Focus on different varieties of “observables” SIGMOD'04

30. First Impressions: Topology bank store k’ a’ a k r b authorize o p mediator store bank ok p2 o1 b2 r1 r2 b1 order1 o2 receipt1 p1 ware- house2 bill2 payment2 ware- house1 payment1 order2 receipt2 bill1 ware- house1 ware- house2 Two common approaches: • Peer-to-peer • Mediated, or “hub and spoke” SIGMOD'04

31. First Impressions: Enactments authorize store bank ok order1 receipt1 bill2 payment2 payment1 order2 bill1 receipt2 ware- house1 ware- house2 • “Enactment” = the execution of multiple steps in a (composite) service, corresponding to a single instance of a (possibly complex) business process • Nested enactments: one authorize, several orders SIGMOD'04

32. Compositions vs. Complex Individual Services • Re-usability of component parts • For individual services, this is a design goal, but not enforced • For compositions, this is foundational assumption • World view • Components of individual service can “see” the rest of the service, modulo scoping, etc. • Services in a composition have a limited interface to “see” other services (typically via messages only) • Implications on transactional aspects • Management of different enactments • Individual service: Details of enactment management are hidden • Composite service: Need mechanism for associating activities of component services with appropriate global enactment • BPEL uses the phrase “correlation sets” SIGMOD'04

33. Models we describe here Emphasize Individual service Emphasize Composite service Style Model SIGMOD'04

34. Mealy Service Model [Bultan et al WWW’03] • Individual service as a Mealy (finite state) machine: • Input and output messages only • Finite state control • Describes behavioral signatures • Abstraction of WSCL • Composition: connecting related services !r2 ?o2 !b2 !b2 authorize store bank !r2 ok ?p2 ?p2 !r2 order1 receipt1 bill2 payment2 payment1 e order2 bill1 receipt2 ware- house1 ware- house2 warehouse2 SIGMOD'04

35. Asynchronous Communication With Queue o1 • Asynchronous, for example, the following channel: ware- house1 store order1 send Order1 … • Queues are FIFO, unbounded length • Can simulate synchronous and also bounded queues send Order1 receive Receipt1 … SIGMOD'04

36. Conversations (an abstraction of enactments) store bank Watcher ware- house1 ware- house2 • Watcher: “records” the messages as they are sent authorize ok payment1 payment2 order2 receipt1 bill2 order1 a k o1 o2 b1 p1 r1 r2 b2 p2 bill1 receipt2 • A conversation is a sequence of messages the watcher sees in a successful run (or enactment) • Conversation language: the set of all possible conversations • What properties do composition languages have? SIGMOD'04

37. Conversation Languages Are Not Regular ?a a !a ?b b !b p1 p2 • CL a*b*=anbn • Composition languages are not always regular • Some may not even be context free • Causes: asynchronous communication &unbounded queue • Bounded queues or synchronous: CL always regular • CLs are always context sensitive SIGMOD'04

38. “Roman” model: An automata-based models with activities[Berardi et. al. ICSOC 03] init search listen cart buy • Abstract behavior of the Service: Client selects next activity Online Music Store • Model of human-machine web services (e.g., Amazon) • Focus on activities Client Do until Client selects “End” • Give Client a choice of actions to be performed • Wait for Client choice • Perform action chosen by Client on-line music store Service SIGMOD'04

39. Roman Model: Automata representation init search listen cart buy Music store search listen cart search init cart buy search search • Transitions labeled by activities • More abstract than message-based approach • For a given state, the out-edges represent the set of options that will be presented to the user SIGMOD'04

40. Delegator: Activity-based FSM annotated with delegations Roman model: Composition buy cart listen search init Delegator for music store search Web ??? ??? listen cart Juke search init Web Web Web cart buy Web Bank search search Web Web Web store Juke Bank search cart init listen buy search SIGMOD'04

41. Data Driven Web Service [Deutsch et al PODS’04] Home page(HP) Name passwd login cancel Customer page(CP) Error message page(MP) Desktop Myorder laptop back Desktop Search(SP) laptop Search(SP) Desktop search Ram: Hdd: Desktop search Ram: Hdd: Display: Past Order (POP) Past Order search search Product index page(PIP) Order status(OSP) Order status Matching products Cancel confirmation page(CCP) Product detail page(PP) Product detail buy Confirmation page(CoP) Order detail • Emphasis is on interaction between control flow and database contents • Transitions resembleDatalog rules and update the database • Rules + DB can be used to simulate control structures database Effects(updatable) Read only SIGMOD'04

42. An Abstract Perspective: Rule-based Control • A hybrid combining automata and logic Condition basedon database query Updates tothe database A complex service: If y then run S If y1 then run S1 : database S If yn then run Sn E’ Effects(updatable) E Read only SIGMOD'04

43. Situation Calculi and PSL: Logics with Actions and Tree-based models • Focus on description of properties, not execution • Models: • Trees whose nodes correspond to atomic actions • “Fluents”: • Propositions (and predicates) which hold between the actions • Used to test pre-conditions, record effects • Vocabulary of PSL (a very rich situation calculus) • Various layers of reified predicates, e.g., • activity (a), activity_occurrence (o), timepoint (t) • occurrence_of (o, a), min_precedes (o1, o2, a) • holds (f, o), prior (f, o) • Activities and occurrences identified using variables and terms (e.g., withdraw(x, y)) • First-order logic, with a family of axioms SIGMOD'04

44. PSL: Simple illustration of the model theory Balance(buyer, 300) Balance(buyer, 295) Balance(buyer, 195) Atomic activities: • Can add constraints, e.g., that w1 must precede w2 • Can use FOL inference or domain-specific reasoning w1 = withdraw (100, buyer) d1 = deposit (100, seller) w2 = withdraw (5, buyer) d2 = deposit (5, broker) init w1 w2 w1 d1 w2 d1 w1 d2 transfer(100, buyer, seller) w2 d2 d2 d1 w2 d1 w1 d2 d2 d2 d1 d1 d2 d1 Combinations of those transfers transfer(5, buyer, broker) SIGMOD'04

45. Expressive power of PSL and Situation Calculi Examples of PSL • Activities as terms: x,y,z activity( transfer(x,y,z) ) • Composition relationships:  x,y,z subactivity(withdraw(w,y), transfer(x,y,z)  a,y ( a = buy_product(y)  x,z subactivity( transfer(x,y,z) , a ) ) • Process description for buy_product  o,x occurrence_of(o, buy_product(x) )  o1,o2,y,z,w,v (occurrence_of( o1, transfer(y,x,z)   occurrence_of(o2, transfer(w,x,v)   subactivity_occurrence(o1, o )   subactivity_occurrence(o2, o ) ) Situation Calculi typically less expressive • Cannot have variables for composite activities/occurrences • Cannot have terms for activities (e.g., transfer(x,y,z)) SIGMOD'04

46. Golog: “Programming” as constraints init w2 w1 w2 d1 w1 d2 d2 d2 d1 w2 d1 w1 d2 d2 d1 d1 d2 d1 • Golog: a “programming language” for the situation calculus • “Constructs” such as • Sequence: 1; 2 • Conditional:if then 1else 2 endif • Loop:while do  endWhile • Interpreted as temporal constraints on permitted paths • E.g., “ w1; w2 ” is satisfied by 3 of the 6 branches • Two-tier “program” specification • First tier: use the “constructs” from above • Identifies a set of possible execution sequences • Second tier: arbitrary constraints • Further restricts set of possible execution sequences SIGMOD'04

47. CTR/CTR-S: Logics specialized to services • Concurrent Transaction Logic (CTR) [Bonner, Kifer ’96] • A logic that extends first-order logic • Three connectives that capture key programming constructs in concurrent transactions • An abstract notion of “update” to shared store • A model theory based on sequences of states • A Horn clause fragment with proof theory • A framework for rules-based specification of “programs”, combined with arbitrary constraints • CTR-Services [Davulcu, Kifer, Ramakrishnan WWW’04] • Targeted at negotiation (once services are discovered) • One more connective, specific to “adversarial” peer services SIGMOD'04

48. CTR Constructs (which are constraints) • A simple workflow • (selected) Constructs and intuition •    :  precedes  in any successful execution •  |  :  and  are to be interleaved and both execute •    : either  or  is true in each successful exec. •  : “isolation” – nothing can interleave with  Temporal constraint b “if e is executed, then b must occur before f ” cond1 a g and cond2 d Trigger or cond3 c “if e is executed and cond4 , then do h ” f e a ( b | ( c ( d ( ef ))) ) g SIGMOD'04

49. CTR: Model Theory and Horn fragment cond1 a g and cond2 d or f cond3 c e • Updates against shared “external” world • Simple example: can use standard relational db updates • Complex example: can use abstract data types as the outside world • CTR models are “multi-paths”, i.e., sequences of paths of (traditional) models ( M1 M2 M3 , M4 M5 , M6 M7 M8 M9 ) • Separations correspond to “break-points”, where other CTR program executions might be interleaved • Horn fragment Bottom_part  ( c ( d ( ef ))) Workflow  a ( b | Bottom_part ) g • There is a proof theory and inference algorithm for Horn fragment b SIGMOD'04

50. CTR-Services • New construct:   -- “opponent’s choice” • Intuition: The external world will choose one of  or  • So,you have to verify your theory against both possibilities • Useful for modeling different alternatives that may arise in a contract, e.g., satisfied  (ship  pay)  (  ship  insurance_payout) • Allows game-theoretic perspective • Need to extend model theory of CTR • New models are sets of m-paths • Corresponds to the multiple possibilities created by  • As with CTR, a Horn fragment, a proof theory for it, and an inference algorithm SIGMOD'04