Allocation in Application Layer Networks

1. Exploring Decentralized Resource Allocation in Application Layer Networks T. Eymann, M. ReinickeAlbert-Ludwigs-University, Freiburg (DE) O. Ardaiz, P. Artigas, L. Díaz de Cerio, F. Freitag, R. Messeguer, L. Navarro, D. RoyoUniversitat Politècnica de Catalunya, Barcelona (ES) CATNET project – Open Research, Evaluation(3/2002-3/2003)

2. Problem and objective • Problem: Provisioning services • Requiring (huge amount of) resources • From large number of computers • CDN, Grid and P2P • Objective: evaluation of decentralized mechanism for resource allocation, based on economic paradigm: Catallaxy. (compare against a centralized mechanism using an arbitrator object) • A concrete case for an application is, for instance, the distributed provisioning of web services for Adobe’s Acrobat (for creating PDF files) in an Akamai-like application layer network.

3. Application Layer Networks (ALN) • Application layer networks are software architectures that allow the provisioning of services requiring a huge amount of resources by connecting large numbers of individual computers. They are built over a base network that is used to support this second network, “layered” upon the underlying infrastructure. • Motivation: • ALN have dynamic demands • Deployment/Allocation Requirements: • Programable Infrastructure: • Nodes with BW, Storage & Processing Resources. • Deployment/Allocation Mechanisms: • Resource Allocation Algorithm, ….

4. ALN Lifecycle • Phases: • Deployment: initial positioning of resources. Deployment can also be economically modeled, although we treat as if done. • Allocation: main focus here. • Allocates resources for the demands. • Changes resource locations: • Migrate • Clone

5. Catallaxy Basics • Catallaxy is an alternative word for “market economy” (Mises and Von Hayek of the Neo-austrian economic school) • “Fundamentally, in a system in which the knowledge of the relevant facts is dispersed among many people, prices can act to co-ordinate the separate actions of different people in the same way as subjective values help the individual to co-ordinate the parts of his plan.” (Friedrich A. von Hayek, The Use of Knowledge in Society, 1945) • “The Market” as a technically decentralized, distributed, dynamic coordination mechanism • Adam Smith’s “invisible hand” • Hayek’s “spontaneous order” • Walras’ “non-tâtonnement process”

6. Catallaxy • Coordination mechanism for systems consisting of autonomous decentralized devices. • Based on constant negotiation and price signaling • Based on efforts from both agent technology and economics • Agents are able to adapt their strategies using machine learning mechanisms • Evolution of software agent strategies, a stabilization of prices throughout the system and self-regulating coordination patterns • Earlier work has used economic principles for resource allocation in distributed computer systems, but most of these approaches rely on using a centralized auctioneer

7. Spontaneous order of the participants „Unplanned result of individuals' planful actions“ (Hayek) Constitutive Elements of the Catallaxy Access to a Market Knowledge about availability of resources is transported through price information Constitutional Ignorance Self-interest and autonomy of participants Ability to choose between alternative actions Learning Dynamic Co-Evolution Income expectations and price relations stabilize development Problems Tragedy of commons Free riding Catallaxy properties

8. Catnet Properties • Agent-based solution is always inferior to analytical optimization • Information • The more information is available, the more accurate are the choices • The more agents, the more information exists • Computation • Computation is fully parallel (no central bottleneck) • Solution always exists in the system (no non-allocated resource)

9. Agents State • Agents genotype: • Acquisitiveness • Satisfaction • Price Step • Price Next • Weight Memory • Reputation • For each service: • Price Distribution • For each negotiation: • Negotiation History

10. Parameters to measure • Social Welfare (SWF): • Sum of all utilities over all participants, in a given timespan • Clients subjectively value SC access • Prices change due to “supply and demand” • Individual utility = transaction price – market value • Also: Response Time (REST), Resource allocation efficiency (RAE), Communication cost (CC), Client-Resource assignment distance.

11. Experimental Simulator • Abstracts from a concrete application and implementation. • Allows „plug-in“ of different „middleware“ resource allocation mechanisms. • Allows easy changes of • Decentralized agent strategies • Centralized allocation mechanisms.

12. Changing node dynamics high networks In an “abstract” simulator , P2P hoc - , ad overloaded Mobile medium CDN networks GRID node density Fixed low medium high CDN P2P Stable A few, powerful A lot, modest GRID Simulation of ALNs ALN

13. Javasim • The Catnet simulator is build over JavaSim, JavaSim is a network simulator based in autonomous components. • Javasim models almost every aspect of a real network: latency, bandwith, lost packets, routing, … • It has some of the more common internet protocols like DV, TCP, UDP, … • So our components can be easily modified to work in the real world changing the middleware to real sockets.

14. R C SC Port 101 Port 102 Port 103 Components • On top of the physical nodes, a number of different software agents are created, which form the application layer network: • Client (C): computer program at host, requests service • Service Copy (SC): instance of service, hosted in a resource computer • Resource (R): host computer with limited storage and bandwidth • Independent on each other at javasim level • Running as programs with a socket on a computer • Configuration made at startup script UDP IP

15. Catallactic Message Flow

16. Components Generic behaviour on messages Using generic functions: - Bargain/RecommendedAction - Price management So changing strategies is easy Particular behaviour on some messages

17. Configuration • We use TCl to set-up the experiments: • Topology • Node configuration: wich components (C/R/SC/MSC) should be on each node. • Application Layer Network initialitzation • Agent parameters: bandwith, price ranges, money balance, genotype, … • Current experiment parameters

18. Output - 1

19. Output - 2 (Catallaxy shows development over time)

20. Output - 3

21. Soundness of Criteria • Interdepencies • SWF and RAE are dependent • Every transaction adds to SWF • More transactions add to RAE • SWF and CC are dependent • Higher CC lowers SWF • SWF and REST are dependent • Higher REST means more transactions • More transactions add to RAE and SWF • SWF captures all costs and revenues • Dependencies are an emergent feature of the system • No direct links have been implemented: economic reasoning works „bottom-up“ in an ACE sense

22. Conclusions • Initial simulation results prove that a decentralized, economic model works better in certain situations. • “Better” is a combination of factors (SWF) • Promising: • Large scale • Dynamic • Saturation

23. Future • Future research work: • Agent technology layer • Application-specific layer • Both are linked in a feedback loop. • Also: • A lot of influencing parameters apart from Density and Dynamism, not fully evaluated due to time constraints.

24. END • Any questions? • More info on http://research.ac.upc.es/catnet/