Working Notes on Protocol Centric Compositional Architecture: Section 1 – Basics of Compositional Architecture by Robert

1. Working Notes on Protocol Centric Compositional Architecture: Section 1 – Basics of Compositional Architecture by Robert J. DuWors 04 / 26 / 2005

2. Innovating the next Dominant Design for Large Scale Distributed Systems Section 1: Compositional Architecture Need: Architectural understanding of what we really do in practice everyday on the Web and to jump start better architectures for more effective solutions. Solution: Compositional Architecture is the first architectural framework as flexible as the Web itself and offers a powerful framework going forward. Section 2: Protocol Translating User Agents and Models Need: Provision of well structured customized functionality and unique User Experience for specialized User Community markets at competitive cost. Solution: Application Level Protocols (ASLP) link Service Oriented Models (SO-M) that extend core product functionality flexibly and inexpensively, to Service Oriented User Agents (PTUA) that tailor User Experience on the fly. Section 3: Distributed Capability Architecture Need: Trustworthy security from top to bottom of all architectural levels without too much or too little securing of highly scalable, dynamic, distributed systems. Solution: Permission Rule Based Capability model for coarse through fine grained control, and for scalable, adaptable “just right” security controls from top-to-bottom and end-to-end: a big step beyond the currently dominant but troubled Access Control List (ACL) model.

3. Compositional Architecture: The Great Leap Forward • Opportunity: • Gain major competitive advantage from an innovative architectural framework that describes what we • do in practice every day on the web and propels product development going forward. • Needed: Anew dominant design to kick start the next generation of distributed system architecture • Solution: Compositional Architecture • Challenge • Distributed databases replication as the central mechanism of distributed system design failed in the 1980s • Boeing, GE, etc • The Distributed Object Orientation Approach fell short in the 1990s • CORBA is moribund and OSF RPC/DCOM/COM+ is of limited utility in a highly proprietary environment • ISO Open Distributed Processing (ODP) went nowhere • Distributed Objects work best, if at all, for relatively tightly coupled applications in LAN environments • Distributed Objects form really hideous de facto protocols at application level (multiple method • invocations running through a “Sea of Objects” that are almost never documented as such) • The original HTML hypertext model is worn out in the 2000s (1st decade – what do we call this period? The 00s?) • the Web involves a great deal of computation and persistence as well as display dynamically delivered • but the URL/URI did break wide open the notion of linkage in distributed systems (Internet style) • The line between “data” and “executable” has been irretrievably breached • network resources are freely distributed, combined into new entities that are executed and used, and • recombined as needed • there are precious few programming languages or data models with this ability and none that can deal • with the rich interaction of multiple technologies on the Web today, e.g. HTML, CCS, Javascript, XML, • XLST in one dynamically constructed entity (“transclusion”) such as a Protocol Translating User Agents

4. A Quick Introduction to Compositional Architecture

5. Compositional Architecture in a Nutshell: A Sneak Preview • Compositional Architecture describes distributed systems in which: • the basic entities are composed and recomposed out of network resources as needed on the fly (dynamically) • all entities communicate with each other exclusively by means of appropriately defined “by value” protocols • “over the wire” for basic retrieval, transfer, loading and composition of entities, and expressing the application • level semantics of interaction between entities • the entities in Compositional Architecture are made out 4 elemental types (fundamental units): • “Earth” - Units of Persistence (uP) • “Air” - Units of Exchange (uE) • “Fire” - Units of Computation (uC) • “Water” - Units of Transduction (uT) • the basic rules of Compositional Architecture govern how these 4 elemental types can be combined and • transmuted • URIs lubricate the ability to reach out to network resources by means of the appropriate references and protocols: • to retrieve the network resources as needed • or to make use of them in place as appropriate • URIs are to Compositional Architecture as symbol tables are to compilers and loaders (linkage mechanism) • blame the explosion of Compositional Architecture on the wide spread use of URIs! “Transclusion”! • Computational Architecture replaces the long outdated Hypertext model as the dominate Web model • Because so much dynamic composition is going on: • Compositional Architectures often employ conventional multi-level “uses” relationships between layers in • “stacks” • but Compositional Architectures distinctively also make extensive use of “meta” relationships (“describes” or • “defines”) between layers in “meta-meta stacks” ala the 4 level UML meta-meta layers • Compositional Architectures make extensive use of three types of linkage binding: • Early binding - pre-defined linkages established well before use • Late binging - pre-defined linkages established at the time of use • Meta-binding - rules for creating the entity itself & defining linkages at the time of use

6. Compositional Architecture in Context: A Further Sneak Preview • Compositional Architecture (CA) is a dynamic End-to-End network architecture • Subsumes other common architectures: Peer-to-Peer, Client-Server, Star, n-Tier • Subsumes other common paradigms: Object Orientation, File Processing, and Relational Data Model • CA is a real architecture, not an overgrown Software Engineering technique • CA is not in direct conflict with OO, but extends beyond the constricting limits of OO • Object Orientation corsets data and computation to the point of strangulation: information hiding is evil! • CA is a genuine “post-OO” paradigm • CA emphasizes the dynamic as much as the static • in entity relationships • in meta-level relationships • CA overthrows 60 years of “data versus executable” • in most software languages and methodologies, data and computation are segregated or rigidly combined • CA introduces meta-binding in addition to early and late binding • fits naturally with Document Oriented Design containing both data and business rules in one document • promotes higher level knowledge representations • CA is uniquely suited for Distributed Computing • highly dynamic: topology independent and self constructing • universal: technology independent and can describe anything that can be built • equally friendly with any imperative, OO, functional, logic, or temporal frameworks • the only theoretical architecture that encompasses what we do in practice every day on the Web! • Protocol Translating User Agents (PTUA) / Service Oriented Models (SO-M), and the Distributed Security Rules • Capability models follow naturally

7. Physical Network Topology IP Transport Level Topology Application Client Server Topology A Universal Model of Distributed Computing and the 4 Fundamental Types • A network is composed of nodes, and edges called links that represent communications channel (“connectivity”) linkage between the nodes • A directed graph defines which nodes are connected to which nodes by links, and is known as the network’s • topology: • Beware: perceived network topology typically changes by level of abstraction, e.g, • A network diagram (graphic) provides a (partial) abstract representation of network resources: • traditionally nodes represent entities of some kind, e.g. database servers, routers, “objects”, etc. • links represent communications channels in the Information Theory sense of the word. • network topology can be highly dynamic, but usually isn’t, except possibly in networks with a • large number of mobile devices, or some kinds of application level network services, e.g. IM. • A Distributed Computing System is a network that supports four fundamental units (types): • All four types can reside at a node: • Units of Computation (roughly “programs”, “processes”, “scripts” and “rules”), • Units of Persistence (roughly “data”, “memory” and “archives”) • Units of Exchange (roughly “messages”) • Units of Transduction (roughly “inputs-outputs” including the “User Interface”). • Only the Unit of Exchange can move across links between nodes while carrying potentially all four • fundamental units within it.

8. The Four Fundamental Units of Distributed Computing Fundamental Units Abbreviation Description Action Type Interaction Units of Computation “programs, scripts and rules” uC Can change their own state (compute) and the state of all fundamental units. Can only be active when resident at a node. Can send uE outward from node, and receive uE inward to node, across communications channels (links). Can send uT outward from edge nodes (export) or receive uT inward (import) at edge nodes. Active • Can Create, Read, Update, and Destroy (CRUD) all fundamental units: • uC, uP, uE, uT • Can be persisted by uP • Can be carried by uE • Can put in and take out all fundamental units from uE • Can import and export uT Units of Persistence “data and memory” uP Preserves unaltered state of all fundamental units until externally transformed to new state or destroyed by uC. Passive • Can make persistent all units: • uC, uP, uE, uT • CRUD by uC • Can be carried between nodes by uE Units of Exchange “messages” uE Carrier mechanism between nodes: information in the form of the fundamental units that can be sent or received from node to node. Can only be carried across links between nodes. Can only be filled with fundamental units and transmitted, or received and emptied by uC at each node. Passive, except to cross edges • Can carry all fundamental units: • uC, uP, uE, uT • CRUD by uC • Filled with fundamental units and emptied by uC at each node Units of Transduction “I/O including UI” uT Define the “inside” from the “outside of the network: represents information in the form of all fundamental types destined to or arriving from outside of the network at edge nodes. Can only sent or received by uC at edge node. Can only be created by uC and consumed externally (export) or created externally and received by uC (import). Carried inward and outward by some from of channel that is NOT internal to the network, i.e. not by a network link. Passive, except to import or export externally to the network • Can contain all fundamental units: • uC, uP, uE, uT • Can be created by entities external to the network (import) • Can be created uC (export) • Can be received (import) or sent (export) from edge node by uC • Can be read and updated by uC • Can be persisted by uP • Can be carried by uE • Can be consumed by entities external to the network (export) • Can be destroyed uC (import)

9. The Four Fundamental Elements of Distributed Systems Architecture Applied to Protocol Translating User Agents • Units of Computation (uC) • Units of Persistence (uP) • Units of Exchange (uE) • Units Transduction (uT) Network Boundary Logical Protocol Translating User Agent External World Physical Device Physical Display / Output Signal Application Protocol Handler uT uE carrying uP and uC User Inteface Translation of Transduction uP and uC Physical Input Signal Outside World

10. What’s the Big Deal? • “IT architecture‘s attention shifts from the fixed structure of functionality towards active composition of functionality” • Two basic concepts: Dynamic Composition and Protocols. • Unlike Actors and Distributed OO, nodes are NOT classic self contained and bashfully self hiding Objects. • Nodes represent network resources that can hold (be bound to) the fundament units whose content are assumed to be fully visible subject only to the constraints of security policies which enforce whole or partial opaqueness. • Thus nodes are containers whose total content is composed dynamically (composition as a function of time) out of ALL fundamental units: Units of Computation and Units of Persistence with Units of Exchange and Units of Transduction: • node total stateT: = (uC1T & uC2T … & uCnT) & (uP1T … & uPmT) & (uE1T … & uEpT) & (uT1T … & uToT) • where & is the composition operator (“transclusion” in Web Architecture) • Messages carry much more than just “data” but also “program and rules” and “I/O” and even other “messages”. • Messages are not rendezvous tuples like LOTOS or memory tuples like Linda which are major alternates to the Distributed OO model (although both of tuple models do allow their elements to be uC or uP or uE or uT!) • Distributed System messages (uE) can carry any mixture of uC & uP & uE & uT. • The central use of composition necessitates that the contents of each fundamental unit, especially the Units of Computation, are NOT “hidden”, but rather are accessible to be assembled on the fly and immediately used as needed • Protocols (uE + the rules that uC use to produce, transmit, and consume uE) are the means to get the content knowledge contained in the fundamental units around the network for integration (composition) and use at the appropriate node at the appropriate time. • This model handles particularly well distributed dynamic rule assembly and documents with mixed data and rule contents thereby allowing application “rule flow” as readily as application “data flow” e.g. • a JSP (uC) assembled web page (uE) using multiple sources (uP and others) that is sent out with the application content (uP etc.) embedded in HTML (uT) and in Javascript (uC) to a browser (uC). • an XML document (uP) that contains elements some of which are “data” (uP) and some of which are “rules” (uC). • the Dynamic Capability Security model in these notes makes extensive use of assembling security rules (uC) from multiple sources (using uE) on the fly to create an integrated rule set (a new uC) for immediate execution. • Gosling’s original idea of “executable” postscript messages (uC in uE) in the defunct NEWS windowing system.

11. What’s the Big Deal? –cont’d • Knowledge Representation and Automation – The Holy Grail of IT • Units of Computation, Units of Persistence, and Units of Transduction are fundamental to the knowledge representation and automation of all applications • Units of Exchange hint at the existence of the network and may or may not be fundamental to the knowledge • content of an application • Networks of nodes and links do not need to be considered part of the fundamental knowledge content of the • application, although in some applications they might be • The 4 fundamental units are the alpha and omega of all IT architecture and design • they are either directly expressed or readily derived from the requirements, which after all is the meta- • knowledge representation of the desired target application • Architectural Design establishes and Detailed System Design further evolves the details and the binding • rules (time and placement) of the fundamental units • all knowledge contained in the fundamental units is fully visible unless explicitly hidden by security policy • new knowledge is formed out of composing the fundamental units together, which is a very fluid view of • form and function (much better than what Frank Lloyd Wright had available to use as his physical units of • construction!) • promotes separation of an explicit end user intelligible knowledge layer from infrastructure layers • can be mapped to less expressive and more restrictive models such as Object Oriented and Relational • Data Models – but it should be possible to map the fundamental units back out for system refactoring or • re-implementation using entirely different design and implementation techniques • like fractals, the fundamental units are self similar at all scales from the most expansive Enterprise • Architecture to the darkest corners of PERL programming • the fundamental units fully exploit the potential of the von Neumann architecture that IT has avoided for • the last 60 years, i.e. the interaction and transforming power of “weaving together” memory, computation, • data transfer, and input-output • “rule flow” applications can be built as easily as “data flow” applications for highly flexibility

12. What’s the Big Deal? –cont’d • Architecture is the answering of Application Domain Wants with IT Solutions • Wants and Solutions have a non-linear feedback relationship: • Changes in Wants, triggers changes in Solutions. • Changes in Solutions (in particular disruptive technologies), induces changes in Wants: • “Perceived IT Technology shapes Requirements and Requirements shape • desired IT Technology.” • Therefore, any linear IT methodology is a crock (in particular the Waterfall model). • n.b. True innovations wreak disruptive havoc on both pre-established Wants and Solutions, such as • when existing business and technical models are supplanted by new ones. • Architecture must model the application domain, the IT solution domain, and the mapping between the two: • Almost all current IT tools, techniques, and methodologies fall squarely on the side of modeling the IT • solution domain – thus they are really Software Engineering tools even if they occasionally masquerade • as Architectural tools, e.g. UML and MDA, OO[AD], and DBMS. • We need much better ways to enter into the application domain: • which inevitably means using the knowledge representation appropriate to the application • domain, e.g. applied mathematics for conventional engineering applications. • all of this is subject to Philip Amour’s view of software as the automation of knowledge and • software creation as subject to the laws of the levels of ignorance (another source of non-linearity). • Directions in Architectural Methodology • Define an ontology of the application space (i.e. “the things assumed to exist in the domain of • discourse and the relationships between them”). • Define within the ontology the manner by which entities are reified (made into manipulatable objects) • expressible by the 4 fundamental units of distributed systems. • Define the fundamental structural elements and protocols of the systems by which the entities interact. • Turn all of the above over to the Software Engineers as the top level architecture description for them to build.

13. ANSI/SPARCData Independence Architecture OMG 4-LayerUML Architecture OSI Meta ModelLayers 1 thru 4Architecture Uses Describes and Defines Defines Minimum for OSI Transport Layer 4 M3 MetaMetamodel Meta Object Facility (MOF) User DBMS Views M2 Metamodel CWM, UML Metamodel e.g. Class, Interface, Attribute ConcePTUAl DBMS Schema OSI Network Layer 3 M1 Model 1 Specific UML model e.g. Collection, SymbolTable, PurchareOrder, etc. OSI Data Link Layer 2 Internal DBMS Schema M0 “Objects” 2 Specific Object Instances e.g. specific PurchaseOrder number 4569105 OSI Physical Link Layer 1 Physical DBMS Data Organization 1, 2What many take to be the whole UML story Examples of Increasing Architectural Sophistication from Conventional “Uses” Layered Architecture thru to Full Fledged Meta-Layered Architecture In which each Meta-Layer also defines the Meta-Binding Rules (composition) for the Layer beneath it

14. Comparison of Object Oriented and Compositional Architectures

15. The Architectural Relationship to Buzzwords de Jour Question: What do the following have in common? asynchronous programming, message-driven programming, event-based programming, n-tiered distributed systems and service-oriented architecture, etc. Answer: They are all ways to dance around protocol centric architecture and issues without mentioning protocols! API side protocol issues: Asynchronous programming, message-driven programming, event-based programming Structure of functionality: N-tiered distributed systems (glued together by many protocol definitions!) Vague, but close: Service-Oriented Architecture needs to be extended into application analysis and design in addition to deployment architecture. Note: in the PTUA architecture every element is “service enabled”, in fact all nodes are “service only,” except for a PTUA which is half service oriented on its backend and half User Interface oriented on its front end! Totally missing: Architecture of Compositional Systems Indeterminism: All of the above have the problem of conquering indeterminism, which generally is avoided by software engineers like a scalded puppy running from boiling water! But indeterminism is shared by GUI design (don’t know what the user will do next, and that is viewed in terms of protocol design by the PTUA/M architecture), and Real Time Systems (caveat: Real Time Systems also have the major attributes of continuous systems which are a wholly different sort of beast!), and Data Communications Systems (which simply involve low level “data moving” protocols). The buzzwords “asynchronous programming” and “event-based programming” are the software side of indeterminism stemming from the underlying protocols. In general indeterminism can be addressed by the formal techniques of re-active systems: state machines, Petri nets, process algebras, etc. (Danger, danger, none of these can handle continuous systems which is what make Real Time Systems both “real” and difficult to build, i.e. you have to borrow from the applied mathematics of signal processing interlaced with re-active system behavior.)