Exploration

1. Exploration Chapter 4COMP151 Feb 5 2007

2. Chap 4 • Best-first search • Greedy best-first search • Recursive best-first search • A* search • Heuristics • Local search algorithms • Hill-climbing search • Simulated annealing search • Local beam search • Genetic algorithms • Online Search informed searchsystematic algorithms sections 4.3-4.5today’s focus

3. Review: Heuristic Search • Previous lecture looked at informed search algorithms • heuristics: rules that encode some useful knowledge about the search space • For these algorithms, the result was a path from the start state to the goal state • informed search through systematic algorithms

4. Local Search • Now we’ll look at search wherethe pathis irrelevant; the goal state itself is the solution • State space = set of "complete" configurations • Goal: some configuration satisfying constraints • local search • keep a single "current" state, try to improve it in next state

5. Local Search Example • Problem: put n queens on an n × n board with no two queens on the same row, column, or diagonal • path is irrelevant, just need to reach goal

6. Local Search Example • Robot is standing in parking lot between geology and computer science, with no maps or other knowledge • goal: arrive at Burn’s Tower • variation: robot has sensor that allows it to determine its current distance from Burn’s Tower

7. Local Search Algorithms • maintain current state, move only to neighbor states • benefits: • low memory usage – usually constant • can find solutions in large spaces where systematic search is unsuitable

8. Optimization Problems • Rather than an explicit goal, optimization problems look for states that maximize some objective function • For this lecture, will assume goal is to maximize objective function. Minimization problems are easily converted to maximization functions ( -1)

9. State Space Landscape

10. Hill-Climbing Search: 8-queens Problem • h = number of pairs of queens that are attacking each other, either directly or indirectly • h = 17 for the above state

11. Hill-Climbing Search: 8-queens Problem • A local minimum with h = 1

12. Local Search • State Space Landscape • location = state • height = objective function • global maximum: best state in entire landscape • local maximum: a state in which all neighbor states have a lower height • complete: always finds a goal if one exists • optimal: always finds a global maximum

13. Hill-Climbing Search • Always move to highest neighbor state(best objective function value) • climbing Everest in thick fog with amnesia • greedy local search

14. Ridges in State Landscape

15. Hill-Climbing Search • Problem: depending on initial state, can get stuck in local maxima

16. Plateaux and Shoulders • Basic hill climbing will not take “side-steps” to states with same height. • treats flat spaces as local maxima • Allowing sideways moves can get off flat spaces, but will cause infinite loops on plateux • solution: limit number of consecutive sideways steps

17. 8 Queens Example • Basic Hill Climbing • succeeds on 14% initial states (4 moves) • gets stuck on 86% initial states (3 moves) • Hill Climbing with sideways steps • max 100 sidesteps • 94% success • 21 steps on success, 64 steps on failure

18. Stochastic Hill Climbing • Chose a random uphill move • rather than best uphill move • probability can favor steeper moves • slower ascent, but may find better solutions • Variation: First choice hill climbing • generate random moves, pick first that is an ascent • good when state has many (thousands) of successors

19. Random Restart Hill Climbing • If at first you don’t succeed … • Conduct a series of hill climbing searches from random state states • complete with probability  1 (will eventually generate goal as start state) • expected number of restarts = 1/pwhere p is probability of success(14% for 8-queens  7 restarts)

20. Simulated Annealing Search • Escape local maxima by allowing some "bad" moves but gradually decrease their frequency

21. Simulated Annealing Search • One can prove: If T decreases slowly enough, then simulated annealing search will find a global optimum with probability approaching 1 • Widely used in VLSI layout, airline scheduling, etc

22. Local Beam Search • Keep track of k states rather than just one • Start with k randomly generated states • At each iteration, all the successors of all k states are generated • If any one is a goal state, stop; else select the k best successors from the complete list and repeat.

23. Genetic Algorithms • A successor state is generated by combining two parent states • Start with k randomly generated states (population) • A state is represented as a string over a finite alphabet (often a string of 0s and 1s) • Evaluation function (fitness function) has higher values for better states. • Produce the next generation of states by selection, crossover, and mutation

24. Genetic Algorithms • Fitness function: number of non-attacking pairs of queens (min = 0, max = 8 × 7/2 = 28) • 24/(24+23+20+11) = 31% • 23/(24+23+20+11) = 29% etc

25. Genetic Algorithms

27. Local Search: Continuous Space • Problem: continuous space successor functions return infinitely many states • Solution 1: discretize the neighborhood of each state • Solution 2: Use gradient functions • gradients are vectors that give magnitude and direction of steepest slope (derivatives of the objective function) • (We’ll avoid this issue for now)

28. Online Search • Online algorithms must process data as it is received (rather than having all data before beginning) • Online search is useful in • (semi)dynamic environments where there is a penalty for sitting and thinking too long • stocastic domains • Online search is necessary for explorations problems where states and actions are unknown

29. Online Search • Online search can only be performed by an agent executing actions, rather than a computational process • agent cannot know successor states except by executing actions • Agent knows: • actions(s) – list of actions allowed in state s • c(s,a,s’) – step cost function • goal-test(s)

30. Robot Maze

31. Online Search Assumptions • Agent can recognize states it has visited before • avoids infinite loops • Actions are deterministic • Agent may have a heuristic functione.g. distance to goal • Objective: reach goal while minimizing cost • Safely Explorable space – goal can be reached from any reachable state (no dead ends)

32. Performance of Online Search • Competitive Ratio: total path cost actually traverse vs. cost agent would traverse if it knew state space in advance. • best achievable CR is often unbounded (very high, possibly infinite) • adversary argument • Better to evaluate performance relative to size of state space, rather than depth of shallowest goal

33. Adversary Arguments • Imagine an adversary that constructs state space as agent is moving through it and can adjust unexplored space to create worst case behavior. • The space that would be created by this adversary is a space that would yield worst-case behavior.

34. Adversaries agent’s view adversary’s view

35. Adversaries

37. Online Hill Climbing? • Hill climbing is on-line, since it only keeps on current state • Problem still exists: local maxima • Random restart is not possible

38. Random Walks • Instead of random restart, we can consider random walks for online search • select a possible action at random, give precedence to untried actions • probability of success  1 for finite space

39. Quicksand

40. Online Search with Memory • Store current best estimate of cost H(s) for each visited state. • start with heuristic, update as agent gains experience • H(s)  c(s,a,s’) + H(s’)

42. LRTA* in one dimension