GC16/3011 Functional Programming Lecture 21 Parallel Graph Reduction

1. GC16/3011 Functional ProgrammingLecture 21Parallel Graph Reduction

2. Contents • Sequential evaluator • Parallel graph reduction • Lazy/Strict/Non-strict evaluation • Strictness analysis • Elements of parallel graph reduction • Issues

3. Sequential evaluator • A sequential evaluator • Normal order (lazy) • Algebraic type = tree • Tree reduction • Can make tree into a graph • Graph reduction • Parallel GR: many sequential evaluators all working on one graph • With additional support for synchronisation and communication

4. Making things easier • Turn all functions into combinators • No free variables! • “Lift” all lambdas to top level • Named functions have no embedded lambdas • So no name clashes, no free variable capture • Easy beta reduction! • “supercombinator reduction”

5. Lazy/Strict/Non-strict evaluation • Normal order reduction (“lazy evaluation”) is SEQUENTIAL • Alternative: Applicative order (“strict evaluation”) • Parallel GR CANNOT be sequential! • Does PGR therefore lose benefits of lazy evaluation? • No! As long as it only evaluates those bits of the graph that normal order reduction would have evaluated • “non-strict” evaluation • Use “Strictness Analysis”

6. Strictness analysis • Done at compile time • For (f x y z) which of x, y and/or z does f always evaluate using normal order reduction? • If (f endless y z) = endless, then assume f MUST evaluate x • Halting problem? • Use approximation technique • Abstract Interpretation

7. Elements of parallel GR • Many processors running sequential evaluators • Graph exists in shared heap(s)

8. @ lx 5 @ @ 3 x + Processor Processor Processor Processor Processor Shared heap

9. Elements of parallel GR • Processors • Each evaluates a subgraph to (weak head) normal form • Slightly modified for Inter-Process Communication and synchronisation • Values communicated via graph • Two processors might evaluate the same sub-graph • just wasted effort, cannot affect correctness • But prefer not to waste the effort!

10. Elements of parallel GR • A processor evaluates one or more tasks • Tasks/threads “to be done” in shared task pool(s) • Synchronisation required whenever a task requires the value of a subgraph that is currently being evaluated by another task • First task blocks, waiting for second task to finish • Processor need not block (can start another task)

11. @ lx 5 @ @ 3 x + Shared heap Processor Processor Processor Processor Processor Shared task pool

12. Sparking • Strictness analysis tells us all of the subgraphs that definitely need to be evaluated • Graph is annotated • At start, just one task descriptor is placed in task pool • One evaluator will start evaluating it • Evaluators check annotations and place task descriptors (sparks) into the task pool • Other evaluators will start work on the sparked tasks

13. Issues • How things are represented in the heap • How the evaluators work • Shared memory (including Virtual SM) or distributed memory • Synchronising tasks (e.g. block and resume) • Scheduling, task distribution and load balancing

14. Summary • Sequential evaluator • Lazy/Strict/Non-strict evaluation • Strictness analysis • Elements of parallel graph reduction • Issues