Computer Science 425 Distributed Systems

Computer Science 425Distributed Systems Inter-process Communication & Distributed Objects

Bank Database Example How are “transactions” executed between a client ATM and a bank server? • Bank Database: Think of two simultaneous deposits of $10,000 into your bank account, each from one ATM. • Both ATMs read initial amount of $1000 concurrently from the bank server • Both ATMs add $10,000 to this amount (locally at the ATM) • Both write the final amount to the server • What’s wrong? • The ATMs need mutually exclusive access to your account entry at the server

Bank Database Example How is your bank account variable replicated (and maintained) among the multiple bank servers? • Bank Database: Think of two simultaneous deposits of $10,000 into your bank account, each from one ATM. • Both ATMs read initial amount of $1000 concurrently from the bank server • Both ATMs add $10,000 to this amount (locally at the ATM) • Both write the final amount to the server • What’s wrong? • The ATMs need mutually exclusive access to your account entry at the server

Search in Chord What are “RPCs”? At node n, send query for key k to largest successor/finger entry < k if none exist, return successor(n) to requestor Say m=7 0 N16 N112 All “arrows” are RPCs N96 N32 Who has bad.mp3? (hashes to K42) File bad.mp3 with key K42 stored here N45 N80

Applications e.g., Java RMI, CORBA and events Middleware Request reply protocol layers External data representation Operating System Middleware Layers RMI=Remote Method Invocation CORBA=Common Object Request Brokerage Architecture

Local Objects • Object • consists of a set of data and a set of methods. • E.g., C++ object such as the Chord object • Object reference • an identifier via which objects can be accessed. • A pointer • Interface • provides a definition of the signatures of a set of methods (i.e., the types of their arguments, return values, and exceptions) without specifying their implementation.

Remote Objects • Remote method invocation • method invocations between objects in different processes (both processes may be on the same host). • Similar to Remote Procedure Call (RPC) • Remote objects • objects that can receive remote invocations. • Remote object reference • an identifier that can be used throughout a distributed system to refer to a particular unique remote object. • Remote interface • Every remote object has a remote interface that specifies which of its methods can be invoked remotely. E.g., CORBA interface definition language (IDL)

local C remote E local invocation invocation remote invocation invocation F B local A invocation D Remote and Local Method Invocations Process Object Process Process Local invocations have exactly once semantics Ideally want similar semantics for remote invocations

remote object Data remote interface m4 { m1 implementation m5 m2 m6 of methods m3 A Remote Object and Its Remote Interface Example Remote Object reference=(IP,port,objectnumber,signature,time)

Failure Modes of RMI/RPC Request Execute lost request correct function Reply Channel fails during reply Execute Request Execute, Crash crash before reply Reply Client machine fails before receiving reply Request Execute Crash crash before execution Reply

Invocation Fault tolerance measures semantics Retransmit request Duplicate Re-execute procedure message filtering or retransmit reply No Not applicable Not applicable Maybe Yes No Re-execute procedure At-least-once Yes Yes Retransmit reply At-most-once Invocation Semantics Transparency=remote invocation has same behavior as local invocation [Birrell and Nelson, inventors of RPC, 1984] Very difficult to implement in asynchronous network… whether to keep a history of result messages to enable lost results to be retransmitted without re-executing the operations Whether or not to retransmit the request message until either a reply is received or the server is assumed to be failed when retransmissions are used, whether to filter out duplicate requests at the server. CORBA (ok for idempotent operations) Java RMI, CORBA Sun RPC

server client remote skeleton object A proxy for B object B & dispatcher Request for B’s class Reply Remote reference Communication Communication Remote module reference module module module Proxy and Skeleton in Remote Method Invocation Process P2 Process P1

Remote Reference Module • Is responsible for translating between local and remote object references and for creating remote object references. • Has a remote object table • An entry for each remote object held by the process. • An entry for each local proxy. • When a remote object is to be passed as argument for the first time, the remote reference module creates a remote object reference and adds it to the table. • When a remote object reference arrives in a request or reply message, the remote reference module is asked for the corresponding local object reference, which may refer either to a dispatcher or to a remote object. • In the case that the remote object reference is not in the table, the RMI software creates a new proxy and asks the remote reference module to add it to the table.

Proxy • Is responsible of making RMI transparent to clients by behaving like a local object to the invoker. • The proxy implements the methods in the remote interface of the remote object it represents. • Instead of executing an invocation, the proxy forwards it to a remote object. • Each method of the proxy marshals (i) a reference to the target object, (ii) its own method Id and (iii) its arguments into a request message and sends it to the target, awaits the reply message, un-marshals it and returns the results to the invoker.

Marshalling & Unmarshalling • External data representation: an agreed standard for the representation of data structures and primitive values. • CORBA Common Data Representation (CDR) • Marshalling: the act of taking a collection of data items and assembling them into the external data representation. • Unmarshalling: the process of disassembling data that is in external data representation form.

Dispatcher and Skeleton • A process has one dispatcher and skeleton for each class representing a remote object. • The dispatcher receives the request message from the communicating module. • It uses the method Id to select the appropriate method in the skeleton, passing on the request message. • Skeleton implements the methods in the remote interface. • A skeleton method un-marshals the arguments in the request message and invokes the corresponding method in the remote object. • It waits for the invocation to complete and marshals the result, together with any exceptions, in a reply message.

Remote Method Invocation (RMI) Client Process Proxy object is a hollow container of Method names. Remote Reference Module translates between local and remote object references. Proxy Object B Object A Remote Reference Module Comm. Module Server Process Dispatcher sends the request to Skeleton Object Skeleton unmarshals parameters, sends it to the object, & marshals the results for return Remote Reference Module Comm. Module Dispatcher Object B Skeleton for B’s Class

Generation of Proxies, Dispatchers and Skeletons • In CORBA, interfaces of remote objects are defined in CORBA IDL, and the interface compiler is used to generate the classes for proxies, dispatchers and skeletons. • In Java RMI • the set of methods offered by a remote object is defined as a Java interface that is implemented in the remote object. • The Java RMI compiler generates the proxy, dispatcher and skeleton classes from the class of the remote object.

Binder and Activator • Binder: A separate service that maintains a table containing mappings from textual names to remote object references. • Used by servers to register their remote objects by name and by clients to look them up. E.g., Java RMI Registry, CORBA Naming Svc. • Activation of remote objects • A remote object is active when it is available for invocation within a running process. • A passive object consists of (i) implementation of its methods; and (ii) its state in the marshalled form. • Activation creates a new instance of the class of a passive object and initializes its instance variables. It is called on demand. • An activator is responsible for • Registering passive objects (recording the names of the servers against the names of the passive objects) • Starting named server processes and activating remote objects in them. • Keeping track of the locations of the servers for remote objects it has already activated • E.g., Activator=Inetd, Passive Object/service=FTP (invoked on demand)

Etc. • Persistent Object = an object that survives between between activations of a process. E.g., Persistent Java, PerDIS, Khazana. • Object Location = instead of binding a remote object reference to an IP+port, use a Location Service to translate from a URL to IP(+well-known port). • Allows the object to migrate from host to host • Example: Akamai migrates web objects using the DNS location service

Remote Procedure Call (RPC) • Uniform, reusable, user-friendly, and action based. • Provide a familiar interface for the application developer • Implement the request-reply primitive • Format of the message is standard • Supports code reuse • Client calls for invocation of a procedure at the server machine. • Semantics are similar to procedure calls in local programs • Standard interface, independent of applications • A library of reusable procedures, distributed over all sites.

client process server process Request Reply client stub server stub procedure procedure service client procedure Communication Communication procedure dispatcher module module Client and Server Stub Procedures in RPC

Stubs • Stubs are generated automatically from interface specifications. • Stubs hide details of (un)marshalling from application programmer & library code developer. • ClientStubs perform marshalling into request messages and unmarshalling from reply messages • Server Stubs perform unmarshalling from request messages and marshalling into reply messages • Stubs also take care of communication & invocation

The Stub Generation Process Compiler / Linker gcc Server Program .o, .exe Server Stub .c Server Source .c .h Interface Specification Common Header RPC LIBRARY RPC LIBRARY Stub Generator e.g., SUN XDR e.g., rpcgen Client Stub Client Source .c .c Client Program .o, .exe Compiler / Linker gcc

Finding RPCs Finding An RPC: RPCs live on specific hosts at specific ports. Port mapper on the host maps from RPC name to port# When a server process is initialized, it registers its RPCs with the port mapper (handle) A client first connects to port mapper to get this handle The call to RPC is then made by connecting to the corresponding port CLIENT Client Stub Client Program Comm. Module SERVER Comm. Module Server procedure Dispatcher Server Stub

External Dealer’s computer Dealer’s computer source Dealer Notification Notification Dealer Information Notification Notification provider Notification Notification Notification Dealer’s computer Dealer’s computer Notification Information provider Notification Notification Dealer Dealer External source Dealing Room System [Publish-Subscribe System] e.g, stock market At each dealer: One object per named stock

Event service subscriber object of interest 1. notification object of interest observer subscriber 2. notification notification object of interest observer subscriber 3. notification Architecture for Distributed Event Notification

Optional Slides – Java RMI

Java Class ShapeListServer with main Method import java.rmi.*; public class ShapeListServer{ public static void main(String args[]){ System.setSecurityManager(new RMISecurityManager()); try{ ShapeList aShapeList = new ShapeListServant(); 1 Naming.rebind("Shape List", aShapeList ); 2 System.out.println("ShapeList server ready"); }catch(Exception e) { System.out.println("ShapeList server main " + e.getMessage());} } }

The Naming Class of Java RMIregistry • void rebind (String name, Remote obj) • This method is used by a server to register the identifier of a remote object by name, as shown in Figure 5.13, line 3. • void bind (String name, Remote obj) • This method can alternatively be used by a server to register a remote object by name, but if the name is already bound to a remote object reference an exception is thrown. • void unbind (String name, Remote obj) • This method removes a binding. • Remote lookup(String name) • This method is used by clients to look up a remote object by name, as shown in Figure 5.15 line 1. A remote object reference is returned. • String [] list() • This method returns an array of Strings containing the names bound in the registry.

Java Remote Interfaces Shape and ShapeList import java.rmi.*; import java.util.Vector; public interface Shape extends Remote { int getVersion() throws RemoteException; GraphicalObject getAllState() throws RemoteException; 1 } public interface ShapeList extends Remote { Shape newShape(GraphicalObject g) throws RemoteException; 2 Vector allShapes() throws RemoteException; int getVersion() throws RemoteException; }

Java class ShapeListServant Implements Interface ShapeList import java.rmi.*; import java.rmi.server.UnicastRemoteObject; import java.util.Vector; public class ShapeListServant extends UnicastRemoteObject implements ShapeList { private Vector theList; // contains the list of Shapes 1 private int version; public ShapeListServant()throws RemoteException{...} public Shape newShape(GraphicalObject g) throws RemoteException { 2 version++; Shape s = new ShapeServant( g, version); 3 theList.addElement(s); return s; } public Vector allShapes()throws RemoteException{...} public int getVersion() throws RemoteException { ... } }

Java Client of ShapeList import java.rmi.*; import java.rmi.server.*; import java.util.Vector; public class ShapeListClient{ public static void main(String args[]){ System.setSecurityManager(new RMISecurityManager()); ShapeList aShapeList = null; try{ aShapeList = (ShapeList) Naming.lookup("//bruno.ShapeList") ; 1 Vector sList = aShapeList.allShapes(); 2 } catch(RemoteException e) {System.out.println(e.getMessage()); }catch(Exception e) {System.out.println("Client: " + e.getMessage());} } }

RemoteObject RemoteServer UnicastRemoteObject Activatable <servant class> Classes Supporting Java RMI

Computer Science 425 Distributed Systems