Parallel SAT using Lemma Exchange: Practical Experiences

1. Parallel SAT using Lemma Exchange: Practical Experiences Yinlei Yu Research Seminar Department of Electrical Engineering Princeton University

2. Outline • Introduction and motivation • Previous work • Parallel SAT by lemma exchange • Lemma exchange in practice • Experimental results • Conclusion and future work

3. Outline • Introduction and motivation • Previous work • Parallel SAT by lemma exchange • Lemma exchange in practice • Experimental results • Conclusion and future work

4. CNF Formula j = ( a +c ) ( b +c ) (a’ +b’ + c’ ) Negative Literal Positive Literal Clause What is Boolean Satisfiability (SAT)? • For a given Boolean formula (usually in Conjunctive Normal Form) • Either find a satisfying truth assignment of all variables • or prove there’s no satisfying assignment

5. Satisfied Literal Unsatisfied Literal Unassigned Literal j= (a+c)(b+c)(a’+b’ + c’) a = T, b = T, c is unassigned Basic Definitions for SAT Solvers • Decision • Select a variable in the CNF, assign it as True or False. • Implication • A variable is forced to be assigned to be True or False based on previous assignments. • Unit clause rule • An unsatisfied clause is a unit clause if it has exactly one unassigned literal. • The unassigned literal is implied because of the unit clause.

6. j = (a+c)(b+c)(a’+b’ + c’) Satisfied Literal Unsatisfied Literal Basic Definitions for SAT Solvers (cont’d.) • Boolean Constraint Propagation (BCP) • Using Unit clause rule to imply variables iteratively until no more implication is possible. • Conflict • All literals in a clause are unsatisfied. • Current assignment cannot satisfy the CNF.

7. Why is SAT Important? • SAT is the first established NP-complete problem [Cook71] • Many typical NP-complete problems, like graph coloring can be easily transformed into SAT problems. • Many applications • Electronic Design Automation • Artificial Intelligence • Other Verification Problems

8. Why do We Need Parallel SAT? • Many SAT problems • cannot be solved in reasonable time • Current single thread solvers are still not fast enough • Need to further accelerate the solvers • Structure of SAT problems are easily parallelizable • PC’s and network components are cheap now • Parallelize it!

9. Outline • Introduction • Previous work • Parallel SAT by lemma exchange • Lemma exchange in practice • Experimental results • Conclusion and future work

10. a 1 0 b b 1 0 0 1 c c 1 0 0 SAT Conflict Previous Work – SAT Solvers • [DP60] First SAT solver based on resolution. • [DLL62] Binary tree searching.

11. -V5 V1’+V5 +V2’ +V2 -V2 +V1 Implied variables Decision Variables V1’+V4’ +V2 Previous Assignments +V4 Previous Work – SAT Solvers • GRASP [SS96] – Incorporate Conflict Driven Learning • Conflict clause (lemma) is generated during search. • Adding a conflict clause does not change the problem. • A conflict clause represents the search space that has been pruned. • The clause is useful when this subspace is encountered in future. • CHAFF [MMZ+01] – Efficient Implementation Conflict Clause (Lemma): V1’+V4’+V5 Assignment +V1 +V4 -V5 caused conflict

12. v1 v1 v1 v1 0 0 1 1 0 0 1 1 v2 v2 v2 v2 v2 v2 v2 v2 1 1 0 0 1 1 0 0 v3 v3 v3 v3 1 1 0 0 1 v4 v4 1 0 Space to be searched Subspace 3 Subspace 1 Subspace 2 Parallel SAT- Search Space Partitioning • Böhm [BS94], Zhang[ZBJ96] • Workload balancing • Primitive single thread solvers

13. PP1 PP2 PP3 PP0 Entire Variable set Make decisions Clause_set1 Variable_set1 Clause_set2 Variable_set2 Clause_set3 Variable_set3 Parallel SAT – Data Set Partitioning • [Zhao01] – data set partitioning on a tightly coupled processor array Decisions, Implications and conflicts

14. PP0 PP1 PP3 PP2 Search Space 0 Search Space 1 Search Space 3 Search Space 2 Lemma exchange Parallel SAT – Lemma Exchange • Sinz, Blochinger and Küchlin[SBK01] • Add lemma exchange to supplement the search space partitioning • Only work on shared memory parallel machine

15. Outline • Introduction and motivation • Previous work • Parallel SAT by lemma exchange • Lemma exchange in practice • Experimental results • Conclusion and future work

16. What is Lemma Exchange? • Different solver threads running on the same SAT problem can exchange their generated lemmas to further prune each other’s search space. • For an UNSAT instance: • Prevent one solver from entering space that has already been searched by other solvers • For a SAT instance: • Additional clauses will result in more implications, thus lead to faster finding of the solution.

17. v1 v1 v1 0 0 0 1 1 1 v2 v2 v2 v2 v2 v2 1 1 1 0 0 0 v3 v3 v3 Subspace 1 1 1 1 0 0 v4 v4 Subspace 2 1 0 v5 Subspace 3 v5 0 0 v7 0 How Can Lemma Exchange Help? • Can prune more search space than normal search space partitioning v1’+v5+v7 v7=1

18. Possible Issues and Solutions • Communication cost is much higher than search space partitioning – needs to transmit the lemmas • Shared memory • Broadcasting (using IP Multicast protocol) • Have a separate computer (master) for lemma relaying (by TCP) • Selectively transmitting lemmas • More clauses will reduce the speed of BCP and the same lemmas may be generated multiple times • Periodically delete irrelevant clauses • Generate a hash based clause database on master

19. Outline • Introduction and motivation • Previous work • Parallel SAT by lemma exchange • Lemma exchange in practice • Experimental results • Conclusion and future work

20. Lemma Exchange: Limitations of Previous Work [SBK 01] • Limited to lemma exchange with search space partitioning • Need to examine general use of lemma exchange with independent solvers. • Limited lemmas (very small clauses) for exchange • Need more experimentation to determine • Limited to shared memory based lemma exchange • Need to examine different types of communication schemes • LAN based lemma exchange • Broadcasting • Have a separate computer for lemma relay

21. Controller Master PP0 PP1 PP2 PPn PP0 PP1 PPn PP2 Whole Search Space Whole Search Space Whole Search Space Whole Search Space Whole Search Space Whole Search Space Whole Search Space Whole Search Space IP multicasting network Control Info Exchanging Lemmas Scheme of LAN Based Lemma Exchange • Based on the single thread Chaff SAT solver[MMZ+01]. • Every solver has the entire clause database and searches the whole space independently with different starting points. • Every lemma is sent to all the other solvers by IP multicasting or via master by TCP. • All solvers will stop if any solver finds a solution or determines UNSAT

22. PP1 PPn PP0 PP2 Whole Search Space Whole Search Space Whole Search Space Whole Search Space Global Shared Memory Exchanging Lemmas Shared Memory Based Lemma Exchange • Exchange clauses via shared memory on the machine • Implemented as multiple threads with global memory or multiple processes with inter-process shared memory • Careful locking scheme so no locking is necessary in lemma exchange

23. LAN vs. Shared Memory

24. Outline • Introduction and motivation • Previous work • Parallel SAT by lemma exchange • Lemma exchange in practice • Experimental results • Conclusion and future work

25. Test Configuration • Benchmarks • From CAD applications • All cases that has run time from 10s to 1000s are selected • Computer Configurations: • LAN Based: • 5 Dell PC servers: • 2 with Pentium III 1.13GHz CPU, 3 with Pentium III 933MHz CPU, all equipped with 1,024MB memory • Connected with dedicated 100BaseT Ethernet switch and separated from external network by router • Shared Memory Based: • 4 UltraSparc II 400MHz CPU SMP with 4,096MB memory • Average Results are shown in following slides

26. Shared Memory Based Experimental Results

27. LAN Based Experimental Results (Broadcasting)

28. LAN Based Experimental Results (Relay)

29. Summary of Results • All three implementations can almost linearly reduce the number of decisions and implications in most test cases. • Sometimes super-linear ratios of implications and decisions are seen. • Speedup is almost linear, but due to overhead, it is usually smaller than the ratio of decisions and implications. • Need to do experiments on a larger number of computers/processors to evaluate the limit of this parallel algorithm.

30. Outline • Introduction and motivation • Previous work • Parallel SAT by lemma exchange • Different communication paradigms • Experimental results • Conclusions and future work

31. Conclusions • Lemma exchange itself is a viable strategy for parallelizing SAT. • In most test cases: • Both LAN based and Shared Memory Based implementations show reasonable speed up. • Almost linearly reduce the number of implications and decisions by lemma exchange.

32. Future Work • Investigate ways for efficiently parallelizing SAT on a large number of processors. • Work on SAT with circuit representations and its parallelization. • Implement parallel version of a Quantified Boolean Formulae (QBF) solver.

33. Reference [DP60] M .Davis, H. Putnam, "A computing procedure for quantification theory", J. of ACM, Vol. 7, pp. 201-214, 1960 [DLL62] M. Davis, G. Logemann and D. Loveland, "A Machine Program for Theorem-Proving", Communications of ACM, Vol. 5, No. 7, pp. 394-397, 1962 [Cook71] S. A. Cook, The complexity of theorem proving procedures, Proceedings, Third Annual ACM Symposium on the Theory of Computing, ACM, New York, 1971, 151-158 [BS94] M. Böhm, E. Speckenmeyer, "A fast parallel SAT-solver -- efficient workload balancing.“ Proc. 3rd Int. Symp. on Artificial Intelligence and Mathematics, 1994 [SS96] J. P. Marques-Silva and K. A. Sakallah, "GRASP -- A New Search Algorithm for Satisfiability," Proc. ICCAD 1996.

34. Reference [ZBJ96]  H. Zhang, M. P. Bonacina and J. Hsiang, "PSATO: A Distributed Propositional Prover and Its Application to Quasigroup Problems" J. Symbolic Computation, Vol. 11 pp.1-18, 1996 [MMZ+01] M. Moskewicz, C. Madigan, Y. Zhao, L. Zhang, S. Malik, "Chaff: Engineering an Efficient SAT SolverProc. 38th DAC, 2001. [SBK01] C. Sinz, W. Blochinger and W. Küchlin , "PaSAT - Parallel SAT-Checking with Lemma Exchange: Implementation and Applications" Proc. SAT2001 pp.212-217 [Zhao01] Y. Zhao, S. Malik, M. W. Moskewicz, C. F. Madigan: "Accelerating Boolean satisfiability through application specific processing." Proc. ISSS2001, pp.244-249