Mobility in Distributed Computing

1. Mobility in Distributed Computing With Special Emphasis on Data Mobility

2. Computing Culture Culture - The predominating attitudes and behavior that characterize the functioning of a group or organization.1 Computing Culture MUST align with social culture to be effective

3. Culture and Mobility “People want to be together; but at the same time they want the opportunity for some small amount of privacy, without giving up community”2

4. What is Mobility? • Access to shared resources through multiple “hard points” • Access to shared resources through dynamic “soft points” • Ability to easily move in and out of a network • Access to computing resources regardless of physical location

5. Mobility “through the ages” (1945-1990) • ENIAC • Computer Time Sharing System • ARPANET • PC’s • Laptops • Ethernet

6. Contemporary Mobility(1990 – Present) • Data • Example: Files • Computing • Example: Specialized Processing • Code • Example: Downloadable GUI

7. Presentation Emphasis • Mobility of Data • Two Examples • Coda File System • Bayou Anti-Entropy Protocol

8. Challenges of Data Mobility • Designing the following characteristics of data mobility is not trivial: • Outside Availability • Outside Semantics • Consistency • Concurrency • Even different systems may have different mobility goals (For example….)

9. Transparency (looks like UNIX) Scalability Push Functionality to Clients Avoid System-wide Rapid Change Balance Availability and Consistency Support for devices w/ limited resources High Availability Application-specific resolution of conflicts Application awareness of underlying system (NOT transparent) Goal Comparison Coda Bayou

10. Coda supports disconnected operations on “cached” versions of shared files Upon reconnection, clients and servers “synch” their file variants. The Bayou AEP allows updates to propagate between participants sharing replicas Write operations are stored and propagated pair-wise (even possibly between clients*) General Comparison Coda Bayou

11. Critical Common Concern • Optimistic vrs Pessimistic Replication • Pessimistic avoids conflicts by restriction • Optimistic detects and resolves conflicts if, and when, necessary • It should be noted that data mobility all but requires optimistic replication

12. Coda Disconnected Operation(simplified view) * User explicitly specifies files he/she wants available offline * Coda makes a best-effort attempt to keep those files in cache * On disconnect, user can access files in cache normally * On reconnect, client and server update one another and resolve any conflicts (write-write only)

13. And Now, For Something Completely Different… • The Bayou Anti-Entropy Protocol attempts to solve a more complex problem • Not accidentally, the Bayou AEP is also more complex than Coda • It doesn’t help that the paper “Flexible Update Propagation for Weakly Consistent Replication” is hard to understand

14. Helpful Bayou Papers http://www2.parc.com/csl/projects/bayou/ • The Bayou Architecture: Support for Data Sharing among Mobile Users • Managing Update Conflicts in Bayou, a Weakly Connected Replicated Storage System • Dealing with Tentative Data Values in Disconnected Work Groups

15. Let’s Start at the Very Beginning • Bayou is designed to run in a mobile computing environment with less than ideal network connectivity • Bayou assumes that mobile users want to share their data despite intermittent network connectivity

16. Supporting Devices with Limited Resources Serversstore data (replicas) in database Clientsread/write data PDA are generally only clients Laptops often operate in both roles

17. Achieving High Availability with Significant Concurrency • Any user can read from or write toany copy of the “database” • Because Bayou assumes that partitions can and do happen, it offers no guarantee of timeliness of write propagation • This means that replicated databases are only weakly consistent

18. What is a “write”? Write <timestamp, s_id> Merge Proc. Dependency Check Updates

19. What is the Database? D0 = Null D1 = W1(D0) D2 = W2(D1) : Dn = Wn(Dn-1) Where Di represents the data after Applying write W0 – Wi in order

20. Getting the Right Write Propagation <96, 3> <100,2> <92, 1> <92, 1> <95, 2> <95, 2> <96, 3> <100,2> <100,2> <96, 3>

21. A Few Notes… • The only requirement for timestamps is that they be monotonically increasing at each server • If timestamps are based on rt-clocks, keeping server’s clocks close is best • Bayou servers maintain a logical clocks to timestamp new writes (initially synched with rt-clock, then updated during AEP)

22. Stabilizing AEP • A write is stable when it’s order will never change • Bayou uses “primary commit protocol” • “Primary” server commits a write • Tentative writes always come after committed writes • NOTE: This is somewhat arbitrary

23. Bayou’s Propagation of Committed Writes • Since committed writes are totally ordered by their CSNs, the highest CSN represents the committed portion of the write-log • In an update, committed writes are transmitted (or a commitment notice sent) before uncommitted writes

24. Stabilizing Images Primary Secondary Secondary Secondary

25. More Bayou “Light Reading” • Session Guarantees • Transportable Media Modifications • Write Log Truncation • Server creation/retirement

26. But… • Throughout our discussion of Bayou, we’ve covered conflict resolution only lightly • How does Bayou conflict resolution work? • How does it compare to Coda conflict resolution?

27. In Coda, conflicts are assumed to be small. When they occur, the user is expected to resolve the conflict (once). In Bayou, each write is assumed to have its own conflict resolution suite (the dependency check and merge procedure) Conflict Resolution

28. Now Wait a Minute… “one crucial assumption is that reordering of concurrent updates, either conflicting or non-conflicting, will result in the same updates to the database. This mandates ‘perfect’ conflict-resolving methods, which seems hard to find for a lot of applications.”3

29. Unanswered Questions • Do these solutions solve the “real” problem? Or, are they solutions looking for a problem? Would YOU use either one? • Do they solve problems best left to applications? (The end-to-end question)

30. Beyond Data Mobility • “Agile Application-Aware Adaptation for Mobility”, Satyanarayanan, et al • “Rover: A Toolkit for Mobile Information Access”, Joseph, et al

31. References • http://www.dictionary.com • C. Alexander, “A Pattern Language”, Oxford University Press, New York, NY, 1977, page 831 • http://www.cs.berkeley.edu/~zf/cs262b/bayou.htm