Distributed Services Part 2

1. Distributed ServicesPart 2 Lecture on Persistence and Transactions Walter Kriha

2. Concurrency Service: provide locks to protect objects from concurrent access which might cause inconsistencies Persistence service: store object state transparently (and load it on demand) Transaction Service: preserve object consistence across changes (several objects, distributed, nested or flat) Preserving Object State Preserving object state in a distributed environment with concurrent access, hardware failures etc. is really hard and needs several services to play together.

3. Distributed objects and their state Object or Resource Need concurreny control Stateless Stateful No fields, no global vars, no locking transient persistent Lifetime like server Server independent lifetime A heated debate concerns the use of stateful objects in distributed systems (like EJB). Some people like Roger Sessions claim that only the stateless version can scale (www.objectwatch.com)

4. Mechanisms for Persistence • Serialize object into a file (Java: stream, COM: structured storage). Does not offer protection against concurrent access. • Use an SQL Driver to store object state. Suffers from “impedance mismatch” and needs to control locking etc. in the service. • Use an object/relational mapper (e.g. Toplink or EJB/Hibernate) to store object state transparently for the programmer. Just storing an object is simple. Doing this in a way that protects from concurrent access, system failures and across different data stores is much harder.

5. Persistent Object Representations Client Dist. service Data store Factory Factory service table Rep. row Proxy mapping The real storage object lives in a data store and uses data store concepts for storage, e.g. a row in a table. The service works with object representations (“Incarnations” according to Emmerich) and provides the illusion of a persistent object to clients

6. Persistent Object Mapping Object view Data store view Class X { Int fieldA; String fieldB; } Create Table Y, 1 integer (primary key) , 2 string Mapping specification: Class X to table Y fieldA to column 1, tagged as primary key fieldB to column2 The key to persistent mapping is meta-information. It is used to generate both the object representations for a service and the code necessary for the data store to store the objects with its own mechanisms and objects. Enterprise integration software also specializes in this kind of mapping.

7. Object Mapping Approaches Object view Create Table Y, 1 integer (primary key) , 2 string Class X { Int fieldA; String fieldB; } Create Table Z, 0 Type of object 1 integer (primary key) , 2 string 3 integer (only derived) Create Table Z, 0 foreign key into Y 1 integer (primary key) , Class Y extends X{ Int fieldC;} what if more derived classes come? two table accesses needed Inheritance creates difficult problems for table mapping. Either performance or flexibility suffer. EJB e.g. does not allow inheritance. A special problem is the extension of a type (class), i.e. to determine all the objects of a type.

8. Persistence mapping in distributed systems CiCs Client Dist. service Procedural mapping OO-DB Rep. OO mapping Rep. table row Rep. O/R mapping A big problem for distributed systems is the fact that many different backends need to be integrated to achieve the transparency effect for a client. This leads to large middleware-frameworks for persistence. The Java Connector Architecture provides an adapter interface for resource managers.

9. Persistent Object Keys • “Foreign keys”: application controlled identity • Artificial keys: data store controlled (e.g. OO database • Service or framework keys: only valid within service and as long as service exists. The main difference between 1) and 2) is that the foreign key is a field (member) of the object while the artificial key is not. Another difference shows up when data should be exported, e.g. to a data warehouse. Artificial keys are meaningless outside the data store.

10. Persistent Object Key and Object Reference If the persistent key is stored in the OR, then the object adapter knows which element to retrieve from the data store . (diagram adapted from van Steen, Tanenbaum) Persistent Key

11. Data Store Session Pooling Dist. service Table Z Rep. row DB session pool Rep. Table Y row Rep. Session (connection) Contract The number of channels to a data store is limited and if an object would directly allocate a session (channel) and not return it quickly, system throughput would become marginal. Also, session creation is expensive (security!). Now either clients ask a pool for a session or the container framework automatically allocates and returns sessions. Problems: timeouts, connection recycling,

12. Why Transactions? • Did your PC crash and you lost the changes you made to a word file? The changes were not DURABLE • Did you move your birthday party to a new location on short notice but couldn’t catch all participants in time so some showed up at the old location and some at the new? Your re-schedule call wasn’t ATOMIC • Did you and a friend work on a shared file on a server and ended up with some of your changes and some of her changes in the file? Your application did not provide ISOLATION between yourself and your friend. • Your friend wants to take a day off and asks you to do some of her work on that day (check out a piece of software, modify it, test it, document it and check it in again). You do it (maybe with some more iterations (;-) and next day she starts a new task. You have observed CONSISTENCY of the tasks.

13. Atomic changes over distributed resources Consistency Isolation from concurrent access Durability of changes Voting algorithm: two phase commit Observation of consistency constraints between objects (or “start consistent, end consistent) Locking mechanisms: 2 phase locking, hierarchical locking Transfer of changes to memory objects to persistent storage Transaction Properties and Mechanisms

14. Transaction API Client: System is in consistent state Begin Transaction Modify objects Commit Transaction System has new, consistent state, all local objects now invalid. The changes are VISIBLE to others. On Error, either the system or the client can do a “rollback” which takes system state back to the beginning of the TA Only in case of a successful commit operation becomes the new state durable and visible to others. Please note that “rollback” really means going back to the beginning COMPLETELY. Theoretically the client does not even KNOW that she tried an operation and even log files would have to be cleaned!

15. Components of distributed transactions Transaction (current) Transactional Servers (objects) Transactional Servers (objects) Transactional Servers (objects) TACoordinator Begin(), commit() Rollback() register Read/write Vote, commit, rollback Transactional Client Read/write/prepare commit ,rollback XA Resource Manager XA Resource Manager XA Resource Manager Every resource that implements the XA interface can participate in distributed transactions.

16. Distributed Two-Phase Commit: Vote Transactional Servers (objects) XA Resource Manager prepare Vote TACoordinator Transactional Servers (objects) XA Resource Manager Vote prepare The only way to achieve “atomic” operations in a distributed setting is to ask all the participants. After a client called “commit()” the TA-Coordinator asks all objects which are part of the TA to vote on either a commit or a rollback. The objects in turn ask the resource managers (e.g. DBs) to “prepare” for a commit. After successful return of a prepare the object AND the resource manager have promised to commit the changes if the coordinator sends a commit.

17. Distributed Two-Phase Commit: Completion Transactional Servers (objects) XA Resource Manager Commit/ rollback Commit/Rollback TACoordinator Transactional Servers (objects) XA Resource Manager Commit/Rollback Commit/ rollback ONLY the coordinator can either commit or abort a TA after the prepare phase. It will call for a commit if the vote phase was successful and all participants have prepared for a following commit. If an error occurred (e.g. a participant was unreachable) the coordinator will call for a rollback.

18. Example of distributed transactions currentTA XAResource1 XAResMgr1 XAResource2 XAResMgr2 Coordinator begin Withdraw money Register resource with coordinator Read/write data place money Register resource with coordinator Work phase Read/write data commit vote 2pc phase 1 Read/write data and prepare vote Read/write data and prepare Do commit Tell resource manager to commit 2pc phase 2 Do commit Tell resource manager to commit

19. Failure models in distributed TA’s Work phase: - A participant crashes or is unavailable in work phase. The coordinator will call for a rollback. - The client crashes in work phase (commit is not called). Coordinator will finally time-out the TA and call rollback. Voting Phase: - If a resource becomes unavailable or has other problems, the coordinator will call rollback Commit Phase: (server uncertainty) - a crashed server will consult the coordinator after re-start and ask for the decision (commit or rollback)

20. Special problems of distributed TA’s • Resources: Participants in distributed TA’s use up many system resources due to logging all actions to temporary persistent storage. Also considerable parts of a system may get locked during a TA. • Coordinator – a single point of failure? Even the coordinator must prepare for a crash and log all actions to temporary persistent storage. • Heuristic outcomes for transactions. Under certain circumstances the outcome of a transaction may only follow a certain heuristic because the real outcome could not be determined. (see exercises)

21. Transaction Types: flat TA’s Commit begin Read/ write Rollback Flat TA’s show the all-or-nothing characteristics of transactions best. ANY failure will cause a complete rollback to the original state. If many objects have been handled this can lose quite a lot of work.

22. Transaction Types: nested TA’s Rollback Commit Parent TA begin Read/ write Child TA begin Child TA commit. Child object now VISIBLE to parent Child TA rollback Nested transactions allow partial rollbacks with a parent transaction. A child TA rollback does not affect the parent TA. But a parent TA rollback will return ALL participants to their initial state. Example: allocation of a travel plan: hotel, flight, rental-car, trips etc. The whole TA should not be aborted only because a certain rental car is not available.

23. Transaction Types: long-running TA’s Rollback Commit begin syncpoint Read/ write Read/ write A rollback will only go back to the checkpoint state A problem of long-running transactions is resource allocation as well as the increasing amount of work that would be lost in case of a rollback. Syncpoints move the fallback position forward towards the commit point.

24. Transaction Types: Compensating TA’s Commit begin commit begin undo side-effects of previous TA Read/ write System detects that commit was too early This is like a “logical” rollback Transaction throughput increases if objects become visible sooner –e.g. through a lose interpretation of the ISOLATION property. Now we need to COMPENSATE for the previous TA (which can no longer be rolled back). It depends on the application whether such compensating transactions are possible. Compensating transactions are also hand-coded if no transaction monitor/manager is available.

25. The interplay of transactions and persistence Transactional Service Data store begin() Lock table. Lock row(Key). Read Row-data factory findObject(Key) DB Create object. Load it with data from row read();write(); Store changes temporarily obj Unlock resources Commit() Make changes permanent (commit) obj Lock row. Load Obj. with data from row begin() Store changes temporarily read();write(); obj The quality of the locks held during a TA is defined through “Isolation levels” in the Resource Manager. Please note that at the beginning of a new TA existing objects are RE-LOADED!

26. Transactions and Isolation Levels (1) The ANSI/ISO SQL standard defines four levels of transaction isolation in terms of three phenomena that must be prevented between concurrenttransactions. dirty reads: A transaction reads data written by concurrent uncommitted transaction. non-repeatable reads: A transaction re-reads data it has previously read and finds that data has been modified by another transaction (that committed since the initial read). phantom read : A transaction re-executes a query returning a set of rows that satisfy a search condition and finds that the set of rows satisfying the condition has changed due to another recently-committed transaction. (From the Postgres manual)

27. Transactions and Isolation Levels (2) Make sure that your resource managers and the transaction controller all work with the same level of isolation and that it is appropriate for your application. Higher levels mean lower performance. Do not try to change levels within transactions!

28. Distributed Deadlocks Node2 Wait for Held by B T1 T2 Held by Wait for Node1 Node3 C A T3 Wait for Held by A distributed deadlock does not show locally. It can be detected either by exporting the local wait-for-graph to a global deadlock detection server (SPOF) or distributed through edge chasing algorithms (probing). (from Coulouris et.al. page 535)

29. Implementing Transactions • Recovery file (transaction log) or version files. Reorganization of log file at runtime. • Crash recovery procedures. • Timestamp ordering concurrency control or optimistic concurrency protocol (needs synchronized global time) • Hierarchical or flat two phase commit with nested TA’s • Algorithms for distributed deadlock detection or prevention (edge chasing, wait-for-graph export etc.) • Timeouts for fatal conditions to avoid resource lockup. • Transaction priorities, unique transaction identifiers.

30. Example: Java Data Objects Goals: • Make user defined classes persistence capable in a mostly transparent fashion. Support inheritance. • The JDO Implementation should be able to shield applications from different and heterogeneous data-stores • JDO Object lifecycle fully transactional (with non-transactional options)

31. Java Data Object Types Object or Resource Stateless Stateful transient persistent Non- transactional Transactional Non- transactional Transactional JDO’s support transactional and non-transactional versions of stateful objects

32. JDO Architecture JDO’s are designed to work in a non-managed form (no application server) and a fully managed form. They are supposed to shield applications from different data sources and mapping problems

33. JDO Implementation Using meta-information (XML mapping files) the JDO implementations does byte-code enhancement to make regular Java classes capable of persistence. (class post-processing) Other alternatives are that classes implement the JDO interfaces themselves or that source code post-processing is done.

34. JDO Vendor: sells JDO implementations Connector Provider: a data store vendor who implements the resource adapter interface Application Server/Container Vendor: builds application servers and containers for the J2EE architecture Application Component Provider: builds application functionality in components Application Assembler: combines components from different sources into a new application Deployer: configure assembled components into specific operational environments System Administrator: manages the configuration and administrations of containers, resource adapters and data stores. JDO - Roles The complexity of modern application server environments shines through the roles involved in building and running those. Will this complexity work in large companies?

35. State Diagram of JDO lifecycle commit MakeNonTA MakeTA setValue makeNonTA getValue Commit/rollb. MakeTA Commit/Rollb. setValue setValue setValue MakePersistent commit Rollback rollback DeletePersistent deletePersistent deletePersistent commit commit rollback

36. Resources (1) • W.Emmerich, engineering distributed objects • CVS Manual. A short and very good explanation of source code control and versioning. • Java Data Objects Version 1.0 (www.java.sun.com) Yes, it is a specification but good to read. • Roger Session, Objectwatch, on Stateless servers (www.objectwatch.com) • Enterprise Java Beans Specification Version 2.0, (www.java.sun.com). Quite a big thing (ca. 570 pages). And requires understanding of isolation levels etc.

37. Resources (2) • Colouris et.al., Chapters 12 an 13 • Ken Birman, Building secure and reliable network applications, Chapter 21 (Transactional Systems). • Grey/Reuters, Transaction Processing (The bible of TA’s) • The Postgres manual (for isolation levels) • Don Chamberlain, Universal Database (even though it’s on DB2 and UDB he knows how to explain the database stuff perfectly – easy to read as well!) • Meet the experts: Gang Chen on Transactions. Details of Websphere TA processing for J2EE architecture. With further links. http://www-128.ibm.com/developerworks/websphere/library/techarticles/0502_chen/0502_chen.html

38. Resources (3) • Promise Mechanism in E (www.erights.org) shows how to avoid threads on the application programming level • Java Communicating Sequential Processes. Middleware that implements Hoares CSP in Java. Excellent introduction by Abhijit Belapurkar on http://www.developers.net/node/view/849 (three parts with many links, e.g. on Pi-calculus for mobility, model checker for parallel process networks • Mozart/OZ • Erlang language for telcos. Provides primitives for asynchronous communication