1 / 53

Self-Adjusting Computation

Self-Adjusting Computation. Robert Harper Carnegie Mellon University (With Umut Acar and Guy Blelloch). The Problem. Given a static algorithm, obtain a dynamic, or incremental, version. Maintain a sorted list under insertions and deletions

bpreston
Download Presentation

Self-Adjusting Computation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Self-Adjusting Computation Robert Harper Carnegie Mellon University (With Umut Acar and Guy Blelloch)

  2. The Problem • Given a static algorithm, obtain a dynamic, or incremental, version. • Maintain a sorted list under insertions and deletions • Maintain a convex hull under motions of the points. • Maintain semantics of a program under edits.

  3. 5, 1, 4, 2, 3 1, 2, 3, 4, 5 5, 1, 2, 3 1, 2, 3, 5 Example: Sorting Input: Output:

  4. Dynamic Algorithms • There is a large body of work on dynamic / incremental algorithms. • Specific techniques for specific problems. • Our interest is in general methods, rather than ad hoc solutions. • Applying them to a variety of problems. • Understanding when these methods apply.

  5. Self-Adjusting Computation • Self-adjusting computation is a method for “dynamizing” a static algorithm. • Start with a static algorithm for a problem. • Make it robust under specified changes. • Goal: “fast” response to “small” change. • “Fast” and “small” are problem-specific! • As ever, the analysis can be difficult.

  6. Self-Adjusting Computation • Generalizes incremental computation. • Attribute grammars, circuit models assume static control dependencies. • SAC permits dynamic dependencies. • Combines algorithmic and programming language techniques. • Linguistic tools to ensure correctness relative to static algorithm. • Algorithmic techniques for efficient implementation.

  7. Self-Adjusting Computation • Adaptivity: Propagate the effects on the output of a change to the input. • Selective Memoization: Reuse old results, provided they are valid after change. • Adaptive Memoization: Reuse old results, even though they may not be valid after change.

  8. Model of Computation • Purelyfunctional programming model. • Data structures are persistent. • No implicit side effects or mutation. • Imperative model of change. • Run on initial input to obtain output. • Make modifications to the input. • Propagate changes to the output.

  9. Steps of Execution Stages of Revision (pure/persistent) (impure/ephemeral) Model of Computation

  10. A Simple Example: Map data cell = nil | cons of int £ list and list = cell fun map (l:list) = map’(l) and map’(c) =case c of nil ) nil | cons (h, t) ) cons (h+10, map t)

  11. 2 3 4 12 13 14 Static Version of Map • Running map on the input list … • … yields the new output list

  12. Dynamic Version of Map • To permit insertions and deletions, lists are made modifiable:data cell = nil | cons of int £ listand list = cell mod

  13. 2 3 5 2 3 5 4 Dynamic Version of Map Insertion changes a modifiable:

  14. 12 13 15 12 13 15 14 Dynamic Version of Map We’d like to obtain the result …

  15. Dynamic Version of Map • Can we update the result in O(1) time? • Make one new call to map. • Splice new cell into “old” result. • Yes, using self-adjusting computation! • Adaptivity: call map on the new node. • Memoization: re-synchronize with suffix.

  16. Adaptivity Overview • To make map adaptive, ensure that • Changes invalidate results that depend on the modified value. • Computations dependent on a change are re-run with the “new” value. • Two key ideas: • Make access to modifiables explicit. • Maintain dependencies dynamically.

  17. Adaptive Map data cell = nil | cons of int £ list and list = cell mod fun map (l:list) =mod (let mod c = l in write(map’ c)) and map’ c =case c of nil ) nil| cons (h, t) ) cons (h+10, map t) Allocate new modifiable Write new modifiable Read old modifiable

  18. 2 3 5 4 Adaptive Map Modification to input: Modified cell is argument of map’, which must be re-run.

  19. 15 14 12 13 12 13 12 13 15 Adaptive Map • Associated output is invalidated, and suffix is re-created. Result of map’ written here.

  20. Adaptive Programming • Crux: dependencies among modifiables. • Writing a modifiable invalidates any computation that reads it. • One read can be contained within another. • Dependencies are fully dynamic! • Cells are allocated dynamically. • Reads affect control flow.

  21. Adaptive Programming • Change propagation consists of • Re-running readers of changed cells. • Updating dependencies during re-run. • To ensure correctness, • All dependencies must be accurately tracked. • Containment ordering must be maintained. • Linguistic tools enforce these requirements!

  22. Type System for Adaptivity • The type  mod is a modality. • From lax modal logic. • And therefore forms a monad. • Two modes of expression: • Stable: ordinary functional code, not affected by changes. • Changeable: affected by change, will be written to another modifiable.

  23. Type System for Adaptivity • Elimination form for  mod:let mod x: = s in c end • Read modifiable given by s. • Bind value to x:, evaluate c. • Makes dependencies explicit: • Records read of given modifiable. • Re-run c with new x if changed. • Reads within c are contained in this read.

  24. Type System for Adaptivity • Execution maintains a trace of adaptive events. • Creation of a modifiable. • Writes to a modifiable. • Reads of a modifiable. • Containment is recorded using Sleator-Dietz order maintenance algorithm. • Associate time intervals with events. • Re-run reader within the “old” time interval. • Requires arbitrary fractionation of time steps.

  25. 12 13 15 14 Adaptive Map • Responds to insertion in linear time:

  26. Memoizing Map • For constant-time update, we must re-synchronize with the old result. • Results after insertion point remain valid despite change. • Re-use, rather than recompute, to save time. • Selective memoization is a general technique for achieving this.

  27. Selective Memoization • Standard memoization is data-driven. • Associate with a function f a finite set of ordered pairs (x, f(x)). • Consult memo table before call, update memo table after call. • Cannot handle partial dependencies. • Eg, read only first 10 elements of an array. • Eg, use an approximation of input.

  28. Selective Memoization • Selective memoization is control-driven. • Guided by the exploration of the input. • Sensitive to approximations. • Associate results with control paths. • “Have I been here before?” • Control path records dependencies of output on input.

  29. Memoized Adaptive Map fun map (l:list) =mod (let mod c = l in write(map’ c)) and memo map’ c =mcase c of nil )return (nil)| cons (h, t) ) let !h’=h and !t’=t in return(cons (h’+10, map t’)) Depends only on nil/cons. Depends on nil/cons, head, and tail.

  30. 2 3 5 4 Memoized Adaptive Map • With selective memoization we obtain • Constant-time update after insertion.

  31. Memoized Programming • Selective memoization requires an accurate record of I/O dependencies. • Which aspects of the input are relevant to the output? • Sensitive to dynamic control flow. • Linguistic support provides • Specification of approximations. • Accurate maintenance of dependencies.

  32. Type System for Memoization • Based on S4 modal logic for necessity. • Truth assumptions: restricted variables. • Validity assumptions: ordinary variables. • Modality ! means value is necessary. • !(int £ int) : both components necessary • !int £ !int : either, neither, or both parts needed, depending on control flow.

  33. Type System for Memoization • Key idea: variables are classified as restricted or unrestricted. • Arguments to memoized functions are restricted. • Results of memoized functions may not involve restricted variables. • Elimination form for ! binds an unrestricted variable. • Ensures that relevant portion of input must be explicitly “touched” to record dependency.

  34. Selective Memoization fun map (l:list) = … and memo map’ c =mcase c of nil )return (nil)| cons (h, t) ) let !h’=h and !t’=t in return(cons (h’+10, map t’)) Restricted Unrestricted

  35. Adaptive Memoization • Effectiveness of memoization depends on preserving identity. • Modifiables compare “by reference”. • Copying a modifiable impedes re-use. • This conflicts with the functional programming model. • Eg, functional insertions copy structure. • Undermines effectiveness of memoization.

  36. 12 13 15 2 3 5 Adaptive Memoization • Consider again applying map to: • We obtain the new modifiable list

  37. 2 3 5 4 Adaptive Memoization • Now functionally insert an element:

  38. 12 13 15 14 Adaptive Memoization • Running map on result yields

  39. 22 23 25 24 Adaptive Memoization • Subsequent runs propagate the effect:

  40. 12 13 15 14 Adaptive Memoization • Ideally, we’d re-use the “old” prefix! • But what justifies this?

  41. Memoization and Adaptivity • Consider a memoized “copy” function:fun copy (!m : int mod) = return (mod (let mod x = m in x)) • What happens if we modify m? • Value might change “under our feet”. • But adaptivity restores correctness!

  42. 43 43 43 43 Memoization and Adaptivity • Initial run of copy: • Calls to copy at later stages return “old” cell, which is updated by adaptivity. 17 17 17 17

  43. Adaptive Memoization • Permit inaccurate memoization. • Allows recovery of “old” cells. • Cached result will be incorrect. • Adapt incorrect result to restore correctness. • Use change propagation to revise answer. • Only sensible in conjunction with adaptivity!

  44. Adaptive Memoization fun map (l:list) = … and memo map’ c =mcase c of nil ) return (nil)| cons (h, t) ) let!h’=h inlet ?t’=t in return(cons (h’+10, map t’)) Do not record dependency! Memo match only on nil/cons and head.

  45. 2 3 5 12 13 15 Adaptively Memoized Map • On the initial input … • Map yields the output …

  46. 2 3 5 4 Adaptively Memoized Map • After a functional update, the input is

  47. 12 13 15 Adaptively Memoized Map • Now maps yields the inaccurate result: • Memo matches on head value 2, yielding old result, with incorrecttail. Tail is incorrect! Result of map

  48. 2 3 5 4 2 3 5 4 Adaptively Memoized Map • Restore accuracy by self-assignment:

  49. 12 13 15 12 13 15 12 13 15 14 Adaptively Memoized Map • Change to input propagates to output

  50. Some Results • Quicksort: expected O(lg n) update after insert or delete at a random position. • Mergesort: expected O(lg n) update after insert or delete. • Tree Contraction: O(lg n) update after adding or deleting an edge. • Kinetic Quickhull: O(lg n) per event measured.

More Related