Introduction to Constraint Programming

1. Introduction to Constraint Programming Mohsen Salarrezaei Advanced Linear programming Course Sharif University of Technology Autumn 2010

2. Outline • Introduction • Constraint propagation • Backtracking search • Some application • Global constraints • Symmetry • Modeling

3. The Tutorial • Goal: to provide an introduction • What is constraint programming? • What is it good for? • How does it compare to integer programming? • How easy is it to use? • What is the underlying technology?

4. Combinatorial Optimization • Many, many practical applications • Resource allocation, scheduling, routing • Properties • Computationally difficult • Technical and modeling expertise needed • Experimental in nature • Important ($$$) in practice • Many solution techniques • Integer programming • Specialized methods • Local search/metaheuristics • Constraint programming

5. Constraint programming • Began in 1980s By computer programmers • Constraints Satisfaction Problem(CSP) • Prolog III (Marseilles, France) • CLP(R) • CHIP (ECRC, Germany) • Application areas • Scheduling, sequencing, resource and personnel allocation, etc. etc. • Active research area • Specialized conferences (CP, CP/AI-OR, …) • Journal (Constraints) • Companies

6. Branch and Prune Prune: eliminate infeasible configurations Branch: decompose into subproblems Prune Carefully examine constraints to reduce possible variable values Branch Use heuristics based on feasibility info Main focus:constraints and feasibility Branch and Bound Bound: eliminate suboptimal solutions Branch: decompose into subproblems Bound Use (linear) relaxation of problem (+ cuts) Branch Use information from relaxation Main focus: objective function and optimality Comparison of CP/IP

7. Illustrative artificial example • Color a map of (part of) Europe: Belgium, Denmark, France, Germany, Netherlands, Luxembourg • No two adjacent countries same color • Is four colors enough?

8. enum Country {Belgium,Denmark,France,Germany,Netherlands,Luxembourg}; enum Colors {blue,red,yellow,gray}; var Colors color[Country]; solve { color[France] <> color[Belgium]; color[France] <> color[Luxembourg]; color[France] <> color[Germany]; color[Luxembourg] <> color[Germany]; color[Luxembourg] <> color[Belgium]; color[Belgium] <> color[Netherlands]; color[Belgium] <> color[Germany]; color[Germany] <> color[Netherlands]; color[Germany] <> color[Denmark]; }; Variables non-numeric Constraints are non-linear Looks nothing like IP! Perfectly legal CP OPL example

9. Example constraint systems/languages

10. Strength of CP’s Modelling • Don’t need to linear constraints • Don’t need to special structure • Since there is no need for a linear relaxation, the language can represent much more directly (no need for big-M IP formulations. • It’s very very easy to modeling for everybody(don’t need to OR knowledge)

11. Integer constraints • Usual arithmetic operators: • =, , , < , > ,  , + , , *, /, absolute value, exponentiation • e.g., 3x + 4y  7, 5x3 – x*y = 9 • Usual logical operators: • , , ,  (or “if … then”) • e.g., if x = 1 then y = 2, x  y  z,(3x + 4y  7)  (x*y = z) • Global constraints: • alldifferent(x1, …, xn) • ordered([A,B,C,D]) • cardinality(x1, …, xn, l, u) • minlist([A,B,C,D], 3) • occurrences(…)

12. more example: Sudoku Each Sudoku has a unique solution that can be reached logically without guessing. Enter digits from 1 to 9 into the blank spaces. Every row must contain one of each digit. So must every column, as must every 3x3 square.

13. using CP; int n=9; dvar int x[1..n][1..n] in 1..9; subject to{ forall(i in 1..n) allDifferent(all(j in 1..9)x[i][j]); forall(j in 1..n) allDifferent(all(i in 1..9)x[i][j]); forall(i in 1..3) forall(j in 1..3) allDifferent(all(k in 3*i-2..3*i,h in 3*j-2..3*j)x[k][h]); } Sudoku

14. more example: n Queens • Place n-queens on an n  n board so that no pair of queens attacks each other

15. n Queens : Solution using CP; int n=8; dvar int x[1..n]in 1..n; subject to{ allDifferent(all(i in 1..n) x[i]); forall(i in 1..n-1) forall(j in i+1..n) abs(x[i]-x[j])!=j-i; }

16. Optimization with CP 1) Find a feasible solution x 2) Z=c(x) 3) Add constraint c(x)<z 4)If found a feasible solution x Goto 2) Else Return last z

17. Outline • Introduction • Constraint propagation • Backtracking search • Some application • Global constraints • Symmetry • Modeling

18. ? ? ? ? ? ? ? ? Place numbers 1 through 8 on nodes, where each number appears exactly once and no connected nodes have consecutive numbers Acknowledgement: Patrick Prosser

19. ? ? ? ? ? ? ? ? Backtracking search Guess a value, but be prepared to backtrack Which nodes are hardest to number?

20. ? ? ? ? ? ? ? ? Backtracking search Which nodes are hardest to number?

21. ? ? ? ? ? ? ? ? Backtracking search Which are the least constraining values to use? Values 1 & 8

22. ? ? ? 1 8 ? ? ? Backtracking search Symmetry means we don’t need to consider: 8 1

23. ? ? ? 1 8 ? ? ? Inference/propagation {1,2,3,4,5,6,7,8} We can now eliminate many values for other nodes

24. ? ? ? 1 8 ? ? ? Inference/propagation {3,4,5,6} {3,4,5,6} By symmetry

25. ? ? ? 1 8 ? ? ? Inference/propagation {3,4,5,6} {1,2,3,4,5,6,7,8} {3,4,5,6}

26. ? ? ? 1 8 ? ? ? Inference/propagation {3,4,5,6} {3,4,5,6} {3,4,5,6} {3,4,5,6} By symmetry

27. ? ? ? 1 8 ? ? ? Inference/propagation {3,4,5,6} {3,4,5,6} {3,4,5,6,7} {2,3,4,5,6} {3,4,5,6} {3,4,5,6}

28. ? ? 7 1 8 2 ? ? Inference/propagation {3,4,5,6} {3,4,5,6} {3,4,5,6} {3,4,5,6}

29. ? ? 7 1 8 2 ? ? Inference/propagation {3,4,5,6} {3,4,5,6} {3,4,5,6} {3,4,5,6} And propagate

30. ? ? 7 1 8 2 ? ? Inference/propagation {3,4,5} {4,5,6} {3,4,5} {4,5,6} Guess a value, but be prepared to backtrack

31. 3 ? 7 1 8 2 ? ? Inference/propagation {4,5,6} {3,4,5} {4,5,6} Guess a value, but be prepared to backtrack

32. 3 ? 7 1 8 2 ? ? Inference/propagation {4,5,6} {3,4,5} {4,5,6} And propagate

33. 3 ? 7 1 8 2 ? ? Inference/propagation {5,6} {4,5} {4,5,6} More propagation?

34. Enforcing local consistency: constraint propagation • Here, focus on: Given a constraint, remove a value from the domain of a variable if it cannot be part of a solution according to that constraint

35. Generic arc consistency algorithm ac() : boolean • Q all variable/constraint pairs (x, C) • while Q {} do • select and remove a pair (x, C) from Q • if revise(x, C) then • if dom(x) = {} • return false • else • add pairs to Q • return true revise(x, C) : boolean change  false for each vdom(x) do if  tCs.t. t[x] = v then remove v from dom(x) change  true return change

36. Generic arc consistency algorithm ac() : boolean • Q all variable/constraint pairs (x, C) • while Q {} do • select and remove a pair (x, C) from Q • if revise(x, C) then • if dom(x) = {} • return false • else • add pairs to Q • return true variable x y z domain {1, 2, 3} {1, 2, 3} {1, 2, 3} constraints C1: x< y C2: y < z revise(x, C) : boolean change  false for each vdom(x) do if  tCs.t. t[x] = v then remove v from dom(x) change  true return change

37. Outline • Introduction • Constraint propagation • Backtracking search • Some application • Global constraints • Symmetry • Modeling

38. Constraint model for 4-queens variables: x1, x2 , x3 , x4 domains: {1, 2, 3, 4} constraints: x1  x2 | x1 – x2 | 1 x1  x3 | x1 – x3 | 2 x1  x4 | x1 – x4 | 3 x2  x3 | x2 – x3 | 1 x2  x4 | x2 – x4 | 2 x3  x4 | x3 – x4 | 1 x1 x2 x3 x4 1 2 3 4

39. Q Q Q Q … Q Forward checking (FC) on 4-queens Q

40. Search tree for 4-queens x1=1 x1= 4 x1 x2 x3 x4 (1,1,1,1) (2,4,1,3) (3,1,4,2) (4,4,4,4)

41. Heuristics for backtracking algorithms • Variable ordering (very important) • what variable to branch on next • Value ordering (can be important) • given a choice of variable, what order to try values

42. Variable ordering • Domain dependent heuristics • Domain independent heuristics • Static variable ordering • fixed before search starts • Dynamic variable ordering • chosen during search

43. Variable ordering: Possible goals • Minimize the underlying search space • static ordering example: • suppose x1, x2, x3, x4 with domain sizes 2, 4, 8, 16 • compare static ordering x1, x2, x3, x4vsx4, x3, x2, x1 • Minimize expected depth of any branch • Minimize expected number of branches • Minimize size of search space explored by backtracking algorithm • intractable to find “best” variable

44. Variable ordering: Basic idea • Assign a heuristic value to a variable that estimates how difficult it is to find a satisfying value for that variable • Principle: most likely to fail first • or don’t postpone the hard part

45. Others methods for solving CP • Local search algorithms • The Min-Conflicts Heuristic(MCH) • The GSAT Algorithm • …. • For study more methods see [1].

46. Outline • Introduction • Constraint propagation • Backtracking search • Some application • Global constraints • Symmetry • Modeling

47. CP can hybrid with other OR’s method • create a relaxation of the CP problem in the form of an OR model • Hybrid with branch-and-price algorithms uses CP for “column generation” • Hybrid with Benders decomposition uses CP for “row generation”

49. computational results for combine CP and Branch and price

50. Problems that has been solved by hybrid op CP and branch and price • Generalized Assignment Problem • airline crew assignment • crew rostering • transit bus crew scheduling • aircraft scheduling • vehicle routing problem • network design • employee timetabling • physician scheduling • traveling tournament problem