The Evergreen Project: The Promise of Polynomials to Boost CSP/SAT Techniques*

1. The Evergreen Project: The Promise of Polynomials to Boost CSP/SAT Techniques* Karl J. Lieberherr Northeastern University Boston joint work with Ahmed Abdelmeged, Christine Hang and Daniel Rinehart Title inspired by a paper by Carla Gomes / David Shmoys

2. Abstract We invent a simple game, called the Evergreen Game, which is about generating and solving Boolean MAX-CSP problems. The fallouts from the Evergreen Game are surprising: • Although the game is about constructing and solving MAX-CSP problems, simple, efficient algorithms are sufficient to guarantee a draw. The best game-playing strategy leads to a significant reduction of the huge search space for both formula generation and solving.

3. Abstract • Fallouts (continued) • The Evergreen Game shows us how to systematically translate a CSP formula into a polynomial that is fundamental in playing the game well. • We have some (but incomplete) evidence that those polynomials are useful for efficient MAX-CSP as well as MAX-SAT and SAT solvers.

4. Where we are • Introduction • The Evergreen Game • The Evergreen Player as Preprocessor • Some Experimental Results

5. Problem Snapshot • SAT: classic problem in complexity theory • SAT & MAX-SAT Solvers: working on CNFs (a multi-set of disjunctions). • Boolean CSP: constraint satisfaction problem • Each constraint uses a Boolean relation. • e.g. a Boolean relation 1in3(x y z) is satisfied iff exactly one of its parameters is true. • Boolean MAX-CSPa multi-set of constraints.

6. Introduction • Boolean MAX-CSP(G) for rank d, G = set of relations of rank d • Input • Input = Bag of Constraint = CSP(G) instance • Constraint = Relation + Set of Variable • Relation = int. // Relation number < 2 ^ (2 ^ d) in G • Variable = int • Output • (0,1) assignment to variables which maximizes the number of satisfied constraints. • Example Input: G = {22} of rank 3. H = • 22:1 2 3 0 • 22:1 2 4 0 • 22:1 3 4 0 1in3 has number 22 M = {1 !2 !3 !4} satisfies all

7. Variation MAX-CSP(G,f): Given a CSP(G) instance H expressed in n variables which may assume only the values 0 or 1, find an assignment to the n variables which satisfies at least the fraction f of the constraints in H. Example: G = {22} of rank 3 MAX-CSP({22},f): H = 22:1 2 3 0 22:1 2 4 0 in MAX-CSP({22},?). Highest value for ? 22:1 3 4 0 22: 2 3 4 0

8. Where we are • Introduction • The Evergreen Game • The Evergreen Player as Preprocessor • Some Experimental Results

9. The Game by Example(special case of Evergreen(2,2)) • The Evergreen Game is played by two players, Anna and Bob, that take turns creating and solving CSP formulae and paying each other a percentage of a wager based on the fraction of constraints satisfied. • Let the wager w be 1 million dollars and the constraints limited to Gamma ={OR(x,y), NOT(x)}.

10. The Game by Example • Anna starts by constructing FInitial = • {100: NOT(x), 150: NOT(y), 200: OR(x,y)}. • Bob tries to find an assignment that satisfies the largest possible fraction of constraints. For example, the assignment {x=true, y=false} will satisfy (150+200)/450 approx 0.78. Anna then pays Bob 0.78 million dollars (w*0.78).

11. The Game by Example • Bob now constructs a formula that Anna solves and pays Anna the percentage of the wager that she solved.

12. Now Bob constructs a formula for Anna: {3: NOT(x), 3: NOT(y), 2: NOT(z) 1: OR(x, y), 1: OR(x, z), 1: OR( y, z)} • The best assignment that Anna finds is {x=false, y=false, z=true} which satisfies about the fraction 0.72. • Bob keeps 0.06 million in his pocket.

13. Theorem 1 Game Evergreen(2,2) has polynomial time algorithms Construct(2,2) and Solve(2,2) for Bob so that Bob can achieve a draw even if Anna has unlimited computational resources.

14. The Game Evergreen(r,m) for Boolean MAX-CSP(G), r>1,m>0 Two players: They agree on a protocol P1 to choose a set of m relations of rank r. • The players use P1 to choose a set G of m relations of rank r. • Player 1 constructs a CSP(G) formula H with 1000 variables and gives it to player 2 (1 second limit). • Player 2 gets paid the fraction of constraints she can satisfy in H (100 seconds limit). • Take 1 turn and stop. How would you play this game intelligently?

15. For details • http://www.ccs.neu.edu/home/lieber/evergreen/game-life-science.html

16. Anna’s Objective:inf max problem sat(H,M) = fraction of satisfied constraints in CSP(G)-formula H by assignment M tG = inf max sat(H,M) all CSP(G) instances H all (0,1) assignments M

17. Bob’s Objective Find an assignment that is at least as good as tG : Algorithm Evergreen Player (linear time). tG = inf max sat(H,M) all CSP(G) instances H all (0,1) assignments M

18. Where we are • Introduction • The Evergreen Game • The Evergreen Player as Preprocessor • Some Experimental Results

19. Experiment • We propose to put the Evergreen Player into action as a preprocessor for state-of-the-art SAT and MAX-SAT solvers. • Use Evergreen Player to create a maximal assignment J for an input formula F. • Feed n-map(F,J) to a fast solver.

20. Where we are • Introduction • The Evergreen Game • The Evergreen Player as Preprocessor • Some Experimental Results

21. Results from 2007 Benchmarks • Within the MAX3SAT benchmarks, there are 4 formulae where Toolbar timed out at 1200 seconds. (v70-c700.wcnf ~ v70-c1000.wcnf). Among these formulae, 1 has its ratio gotten worse (0.9985795) and 3 of 4 have  their ratio gotten better with the average being roughly 1.0099673. • Within the 3 MAXCUT benchmarks I've tried, there is one formula where Toolbar timed out at 1200 seconds. This formulae has its ratio unchanged. • Among all the 20 benchmarks I've finished, 5 of them fall into the time-out category.

22. Other results from 2007 Benchmarks • On some benchmarks where no timeout occurred the running time got better (by factors of 2 and 3) in 50 % of the cases with preprocessing. • Preprocessing is very fast (linear).

23. yices: a nice improvement on one of the first examples we tried • Yices without preprocessing:v2000-c8400average time = 888.048average sat ratio = 0.947143 • Yices with preprocessing:v2000-c8400average time = 0.0342615average sat ratio = 1

24. Conclusions • Worth to investigate further. • Suggests a cheap way to parallelize MAX-SAT and SAT solving: Run preprocessed and unpreprocessed version in parallel.

25. Thank you. • The End.

26. Our approach by Example:SAT Rank 2 example 14 : 1 2 014 :       3 4 014 :           5 6 0  7 : 1    3            0 7 : 1        5      0 7 :       3   5      0 7 :   2    4         0 7 :   2         6   0 7 :         4   6   0 14: 1 2 = or(1 2) 7: 1 3 = or(!1 !3)

27. excellent peripheral vision 0 1 2 3 4 5 6 = k 8/9 7/9 Blurry vision • What do we learn from the abstract representation? • set 1/3 of the variables to true (maximize). • the best assignment will satisfy at least 7/9 constraints. • very useful but the vision is blurry in the “middle”. appmean = approximation of the mean (k variables true)

28. Our approach by Example • Given a CSP(G)-instance H and an assignment N which satisfies fraction f in H. • Is there an assignment that satisfies more than f? • YES (we are done), absH(mb) > f • MAYBE, The closer absH() comes to f, the better • Is it worthwhile to set a certain literal k to 1 so that we can reach an assignment which satisfies more than f • YES (we are done), H1 = Hk=1, absH1(mb1) > f • MAYBE, the closer absH1(mb1) comes to f, the better • NO, UP or clause learning absH= abstract representation of H

29. 8/9 7/9 abstract representation 14 : 1 2 014 :       3 4 014 :           5 6 0  7 : 1    3            0 7 : 1        5      0 7 :       3   5      0 7 :   2     4          0 7 :   2         6   0 7 :         4   6   0 H 3/9 0 1 2 3 4 5 6 14 : 2 014 :       3 4 014 :           5 6 0  7 : 1    3            0 7 : 1        5      0 7 :       3   5      0 7 :   2     4          0 7 :   2         6   0 7 :         4   6   0 6/7 = 8/9 5/7=7/9 H0 3/7=5/9 maximum assignment away from max bias: blurry 0 1 2 3 4 5

30. 8/9 7/9 14 : 1 2 014 :       3 4 014 :           5 6 0  7 : 1    3            0 7 : 1        5      0 7 :       3   5      0 7 :   2     4          0 7 :   2         6   0 7 :         4   6   0 3/8 H 0 1 2 3 4 5 6 clearly above 3/4 14 : 1 2 014 :       3 4 014 :           5 6 0 7 :     3            0 7 :         5      0 7 :       3   5      0 7 :   2     4          0 7 :   2         6   0 7 :         4   6   0 7/8=8/9 6/8=7/9 H1 maximum assignment away from max bias: blurry 2/7=3/8 0 1 2 3 4 5

31. 14 : 1 2 014 :       3 4 014 :           5 6 0  7 : 1    3            0 7 : 1        5      0 7 :       3   5      0 7 :   2     4          0 7 :   2         6   0 7 :         4   6   0 8/9 7/9 abstract representation guarantees 7/9 H 7/8 = 8/9 6/7=8/9 6/8 = 7/9 5/7=7/9 abstract representation guarantees 7/9 abstract representation guarantees 8/9 H0 H1 NEVER GOES DOWN: DERANDOMIZATION

32. rank 2 10: 1 = or(1) 7: 1 2 = or(!1 !2) 4/6 4/6 10 : 1 0 10 : 2 0 10 : 3 0 7 : 1 2 0 7 : 1 3 0 7 : 2 3 0 3/6 3/6 abstract representation guarantees 0.625 * 6 = 3.75: 4 satisfied. 0 1 2 3 4/6 4/6 4/6 4/6 5 : 1 0 10 : 2 0 10 : 3 0 13 : 1 2 0 13 : 1 3 0 7 : 2 3 0 3/6 3/6 rank 2 5: 1 = or(!1) 13: 1 2 = or(1 !2) The effect of n-map

33. First Impression • The abstract representation = look-ahead polynomials seems useful for guiding the search. • The look-ahead polynomials give us averages: the guidance can be misleading because of outliers. • But how can we compute the look-ahead polynomials?

34. Where we are • Introduction • Look-forward • Look-backward • SPOT: how to use the look-ahead polynomials together with superresolution.

35. Look Forward • Why? • To make informed decisions • How? • Abstract representation based on look-ahead polynomials

36. Look-ahead Polynomial(Intuition) • The look-ahead polynomial computes the expected fraction of satisfied constraints among all random assignments that are produced with bias p.

37. Consider an instance: 40 variables,1000 constraints (1in3) 1, … ,40 22: 6 7 9 0 22: 12 27 38 0 Abstract representation: reduce the instance to look-ahead polynomial 3p(1-p)2 = B1,3(p) (Bernstein)

38. 3p(1-p)2 for MAX-CSP({22})

39. Look-ahead Polynomial(Definition) • H is a CSP(G) instance. • N is an arbitrary assignment. • The look-ahead polynomial laH,N(p) computes the expected fraction of satisfied constraints of H when each variable in N is flipped with probability p.

40. The general case MAX-CSP(G) G = {R1, … }, tR(F) = fraction of constraints in F that use R. appSATR(x) over all R is a super set of the Bernstein polynomials (computer graphics, weighted sum of Bernstein polynomials) x = p

41. Rational Bezier Curves

42. Bernstein Polynomials http://graphics.idav.ucdavis.edu/education/CAGDNotes/Bernstein-Polynomials.pdf

43. all the appSATR(x) polynomials

44. Look-ahead Polynomial in Action • Focus on purely mathematical question first • Algorithmic solution will follow • Mathematical question: Given a CSP(G) instance. For which fractions f is there always an assignment satisfying fraction f of the constraints? In which constraint systems is it impossible to satisfy many constraints?

45. Remember? MAX-CSP(G,f): Given a CSP(G) instance H expressed in n variables which may assume only the values 0 or 1, find an assignment to the n variables which satisfies at least the fraction f of the constraints in H. Example: G = {22} of rank 3 MAX-CSP({22},f): 22:1 2 3 0 22:1 2 4 0 22:1 3 4 0 22: 2 3 4 0

46. Mathematical Critical Transition Point MAX-CSP({22},f): For f ≤ u: problem has always a solution For f≥ u + e: problem has not always a solution, e>0. 1 not always (solid) u = critical transition point always (fluid) 0

47. The Magic Number • u = 4/9

48. 3p(1-p)2 for MAX-CSP({22})

49. Produce the Magic Number • Use an optimally biased coin • 1/3 in this case • In general: min max problem

50. The 22 reductions:Needed for implementation 1,0 2,0 22 60 240 3,0 2,1 3,1 1,1 2,0 3,0 3 15 255 3,1 2,1 22 is expanded into 6 additional relations. 0