280 likes | 490 Views
Dave Reed. AI as search problem states, state spaces uninformed search strategies depth first search depth first search with cycle checking breadth first search iterative deepening. AI as search. GOFAI philosophy: Problem Solving = Knowledge Representation + Search (a.k.a. deduction)
E N D
Dave Reed • AI as search • problem states, state spaces • uninformed search strategies • depth first search • depth first search with cycle checking • breadth first search • iterative deepening
AI as search • GOFAI philosophy: Problem Solving = Knowledge Representation + Search (a.k.a. deduction) • to build a system to solve a problem: • define the problem precisely i.e., describe initial situation, goals, … • analyze the problem • isolate and represent task knowledge • choose an appropriate problem solving technique we'll look at a simple representation (states) first & focus on search more advanced representation techniques will be explored later
Example: airline connections • suppose you are planning a trip from Omaha to Los Angeles initial situation: located in Omaha goal: located in Los Angeles possible flights: Omaha Chicago Denver Los Angeles Omaha Denver Denver Omaha Chicago Denver Los Angeles Chicago Chicago Los Angeles Los Angeles Denver Chicago Omaha • this is essentially the same problem as the earlier city connections • we could define special purpose predicates, recursive rules to plan route flight(omaha, chicago). flight(denver, los_angeles). . . . route(City1, City2, [City1,City2]) :- flight(City1, City2). route(City1, City2, [City1|Rest]) :- flight(City1, City3), route(City3, City2, Rest).
State spaces • or, could define a more general solution framework state : a situation or position state space : the set of legal/attainable states for a given problem • a state space can be represented as a directed graph (nodes = states) • in general: to solve a problem • define a state space, then search for a path from start state to a goal state
Example: airline state space • define a state with: loc(City) • define state transitions with: move(loc(City1), loc(City2)) • to find a route from Omaha to Los Angeles: • search for a sequence of transitions from start state to goal state: loc(omaha) loc(los_angeles) %%% travel.pro Dave Reed 2/15/02 %%% %%% This file contains the state space definition %%% for air travel. %%% %%% loc(City): defines the state where the person %%% is in the specified City. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% move(loc(omaha), loc(chicago)). move(loc(omaha), loc(denver)). move(loc(chicago), loc(denver)). move(loc(chicago), loc(los_angeles)). move(loc(chicago), loc(omaha)). move(loc(denver), loc(los_angeles)). move(loc(denver), loc(omaha)). move(loc(los_angeles), loc(chicago)). move(loc(los_angeles), loc(denver)).
Example: water jug problem • Suppose you have two empty water jugs: • large jug can hold 4 gallons, small jug can hold 3 gallons • starting with empty jugs (and an endless supply of water), want to end up with exactly 2 gallons in the large jug • state? • start state? • goal state? • transitions?
Example: water jug state space • define a state with: jugs(LargeJug, SmallJug) • define state transitions with: move(jugs(J1,J2), jugs(J3,J4)) • to solve the problem: • search for a sequence of transitions from start state to goal state: jugs(0,0) jugs(2,0) %%% jugs.pro Dave Reed 2/15/02 %%% %%% This file contains the state space definition %%% for the Water Jug problem. %%% %%% jugs(J1, J2): J1 is the contents of the large %%% (4 gallon) jug, and J2 is the contents of %%% the small (3 gallon) jug. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% move(jugs(J1,J2), jugs(4,J2)) :- J1 < 4. move(jugs(J1,J2), jugs(J1,3)) :- J2 < 3. move(jugs(J1,J2), jugs(0,J2)) :- J1 > 0. move(jugs(J1,J2), jugs(J1,0)) :- J2 > 0. move(jugs(J1,J2), jugs(4,N)) :- J2 > 0, J1 + J2 >= 4, N is J2 - (4 - J1). move(jugs(J1,J2), jugs(N,3)) :- J1 > 0, J1 + J2 >= 3, N is J1 - (3 - J2). move(jugs(J1,J2), jugs(N,0)) :- J2 > 0, J1 + J2 < 4, N is J1 + J2. move(jugs(J1,J2), jugs(0,N)) :- J1 > 0, J1 + J2 < 3, N is J1 + J2.
Example: 8-tiles puzzle • Consider the children's puzzle with 8 sliding tiles in a 3x3 board • a tile adjacent to the empty space can be shifted into that space • goal is to arrange the tiles in increasing order around the outside • state? • start state? • goal state? • transitions?
Example: 8-tiles state space • define a state with: tiles(Row1,Row2,Row3) • define state transitions with: move(tiles(R1,R2,R3), tiles(R4,R5,R6)). • to solve the problem: • search for a sequence of transitions from start state to goal state: tiles([1,2,3],[8,6,space],[7,5,4]) tiles([1,2,3],[8,space,4],[7,6,5]) %%% puzzle.pro Dave Reed 2/15/02 %%% %%% This file contains an UGLY, BRUTE-FORCE state %%% space definition for the 8-puzzle problem. %%% %%% tiles(R1, R2, R3): each row is a list of three %%% numbers (or space) specifying the tiles in %%% that row. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% move(tiles([space,T1,T2],[T3,T4,T5],[T6,T7,T8]), tiles([T1,space,T2],[T3,T4,T5],[T6,T7,T8])). move(tiles([space,T1,T2],[T3,T4,T5],[T6,T7,T8]), tiles([T3,T1,T2],[space,T4,T5],[T6,T7,T8])). move(tiles([T1,space,T2],[T3,T4,T5],[T6,T7,T8]), tiles([space,T1,T2],[T3,T4,T5],[T6,T7,T8])). move(tiles([T1,space,T2],[T3,T4,T5],[T6,T7,T8]), tiles([T1,T2,space],[T3,T4,T5],[T6,T7,T8])). move(tiles([T1,space,T2],[T3,T4,T5],[T6,T7,T8]), tiles([T1,T4,T2],[T3,space,T5],[T6,T7,T8])). move(tiles([T1,T2,space],[T3,T4,T5],[T6,T7,T8]), tiles([T1,space,T2],[T3,T4,T5],[T6,T7,T8])). move(tiles([T1,T2,space],[T3,T4,T5],[T6,T7,T8]), tiles([T1,T2,T5],[T3,T4,space],[T6,T7,T8])). . . .
State space + control • once you formalize a problem by defining a state space: • you need a methodology for applying actions/transitions to search through the space (from start state to goal state) i.e., you need a control strategy • control strategies can be: tentative or bold; informed or uninformed • a bold strategy picks an action/transition, does it, and commits • a tentative strategy burns no bridges • an informed strategy makes use of problem-specific knowledge (heuristics) to guide the search • an uninformed strategy is blind
Uninformed strategies • bold + uninformed: STUPID! • tentative + uninformed: • depth first search (DFS) • breadth first search (BFS) example: water jugs state space (drawn as a tree with duplicate nodes)
Depth first search • basic idea: • if the current state is a goal, then SUCCEED • otherwise, move to a next state and DFS from there • if no next state (i.e., deadend), then BACKTRACK %%% dfs.pro Dave Reed 2/15/02 %%% %%% Depth first search %%% dfs(Current, Goal): succeeds if there is %%% a sequence of states that lead from %%% Current to Goal %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% dfs(Goal, Goal). dfs(State, Goal) :- move(State, NextState), dfs(NextState, Goal). • note: Prolog computation is basically depth first search • if goal list is empty, SUCCEED • otherwise, find fact/rule that matches the first goal & replace the goal with the rule premises (if any) • if no matching fact/rule, then BACKTRACK
Depth first search • generally, want to not only know that a path exists, but what states are in that path • i.e., what actions lead from start state to goal state? %%% dfs.pro Dave Reed 2/15/02 %%% %%% Depth first search %%% dfs(Current, Goal, Path): Path is a list %%% of states that lead from Current to Goal %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% dfs(Goal, Goal, [Goal]). dfs(State, Goal, [State|Path]) :- move(State, NextState), dfs(NextState, Goal, Path). • add an argument to the dfs predicate, which "returns" the path • if current state is a goal, then path is just that state • otherwise, make a move and use recursion to find a path from that new state (but must add that state back onto path)
DFS examples ?- dfs(loc(omaha), loc(los_angeles), Path). Path = [loc(omaha), loc(chicago), loc(denver), loc(los_angeles)] ; Path = [loc(omaha), loc(chicago), loc(denver), loc(los_angeles), loc(chicago), loc(denver), loc(los_angeles)] ; Path = [loc(omaha), loc(chicago), loc(denver), loc(los_angeles), loc(chicago), loc(denver), loc(los_angeles), loc(chicago), loc(...)|...] Yes all are legitimate paths, but not "optimal" ?- dfs(jugs(0,0), jugs(2,0), Path). ERROR: Out of local stack WHY? ?- dfs(tiles([1,2,3],[8,6,space],[7,5,4]), tiles([1,2,3],[8,space,4],[7,6,5]), Path). ERROR: Out of local stack WHY?
DFS and cycles • since DFS moves blindly down one path, • looping (cycles) is a SERIOUS problem
Depth first search with cycle checking • it often pays to test for cycles (as in city connections example): • must keep track of the path so far • make sure current goal is not revisited %%% dfsnocyc.pro Dave Reed 2/15/02 %%% %%% Depth first search with cycle checking %%% dfs_no_cycles(Current, Goal, Path): Path is a list %%% of states that lead from Current to Goal %%% %%% dfs_nocyc_help(PathSoFar, Goal, Path): Path is a list %%% of states that lead from the PathSoFar (with current %%% state at the head) to Goal without duplicate states. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% dfs_no_cycles(State, Goal, Path) :- dfs_nocyc_help([State], Goal, RevPath), reverse(RevPath, Path). dfs_nocyc_help([Goal|PathSoFar], Goal, [Goal|PathSoFar]). dfs_nocyc_help([State|PathSoFar], Goal, Path) :- move(State, NextState), not(member(NextState, [State|PathSoFar])), dfs_nocyc_help([NextState,State|PathSoFar], Goal, Path).
Examples w/ cycle checking ?- dfs_no_cycles(loc(omaha), loc(los_angeles), Path). Path = [loc(omaha), loc(chicago), loc(denver), loc(los_angeles)] ; Path = [loc(omaha), loc(chicago), loc(los_angeles)] ; Path = [loc(omaha), loc(denver), loc(los_angeles)] ; No finds all paths that don’t have cycles ?- dfs_no_cycles(jugs(0,0), jugs(2,0), Path). Path = [jugs(0, 0), jugs(4, 0), jugs(4, 3), jugs(0, 3), jugs(3, 0), jugs(3, 3), jugs(4, 2), jugs(0, 2), jugs(..., ...)] ; Path = [jugs(0, 0), jugs(4, 0), jugs(1, 3), jugs(4, 3), jugs(0, 3), jugs(3, 0), jugs(3, 3), jugs(4, 2), jugs(..., ...)|...] Yes finds solutions, but not optimal (SWI uses … for long answers) ?- dfs(tiles([1,2,3],[8,6,space],[7,5,4]), tiles([1,2,3],[8,space,4],[7,6,5]), Path). ERROR: Out of local stack but it takes longer…
Breadth vs. depth • even with cycle checking, DFS may not find the shortest solution • breadth first search (BFS) • extend the search one level at a time • i.e., from the start state, try every possible move (& remember them all) • if don't reach goal, then try every possible move from those states • . . . • requires keeping a list of partially expanded search paths • ensure breadth by treating the list as a queue • when want to expand shortest path: take off front, extend & add to back [ omaha ] [ chicagoomaha, denveromaha ] [ denveromaha, denverchicagoomaha, los_angeleschicagoomaha, omahachicagoomaha ] [ denverchicagoomaha, los_angeleschicagoomaha, omahachicagoomaha, los_angelesdenveromaha, omahadenveromaha ] [ los_angeleschicagoomaha, omahachicagoomaha, los_angelesdenveromaha, omahadenveromaha, los_angelesdenverchicagoomaha, omahadenverchicagoomaha ]
Breadth first search %%% bfs.pro Dave Reed 2/15/02 %%% %%% Breadth first search %%% bfs(Current, Goal, Path): Path is a list %%% of states that lead from Current to Goal %%% %%% bfs_help(ListOfPaths, Goal, Path): Path is a list %%% of states that lead from one of the paths in ListOfPaths %%% (a list of lists) to Goal. %%% %%% extend(Path, ListOfPaths): ListOfPaths is the list of all %%% possible paths obtainable by extending Path (at the head). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% bfs(State, Goal, Path) :- bfs_help([[State]], Goal, RevPath), reverse(RevPath, Path). bfs_help([[Goal|Path]|_], Goal, [Goal|Path]). bfs_help([Path|RestPaths], Goal, SolnPath) :- extend(Path, NewPaths), append(RestPaths, NewPaths, TotalPaths), bfs_help(TotalPaths, Goal, SolnPath). extend([State|Path], NewPaths) :- bagof([NextState,State|Path], move(State,NextState), NewPaths), !. extend(_, []). • bagof(X,p(X),List): • List is a list of all X's for which p(X) holds • equivalent to hitting ';' repeatedly to exhaust all answers, then putting them in a list • ! • exclamation point is the cut operator • it cuts choice points, committing the interpreter to that match
BFS examples ?- bfs(loc(omaha), loc(los_angeles), Path). Path = [loc(omaha), loc(chicago), loc(los_angeles)] ; Path = [loc(omaha), loc(denver), loc(los_angeles)] ; Path = [loc(omaha), loc(chicago), loc(denver), loc(los_angeles)] ; Path = [loc(omaha), loc(chicago), loc(los_angeles), loc(chicago), loc(los_angeles)] Yes finds shorter paths first, but no loop checking ?- bfs(jugs(0,0), jugs(2,0), Path). Path = [jugs(0, 0), jugs(0, 3), jugs(3, 0), jugs(3, 3), jugs(4, 2), jugs(0, 2), jugs(2, 0)] ; Path = [jugs(0, 0), jugs(4, 0), jugs(1, 3), jugs(1, 0), jugs(0, 1), jugs(4, 1), jugs(2, 3), jugs(2, 0)] ; Path = [jugs(0, 0), jugs(0, 3), jugs(3, 0), jugs(3, 3), jugs(4, 2), jugs(4, 2), jugs(0, 2), jugs(2, 0)] Yes finds shorter paths first, but slow (& looping is possible) ?- bfs(tiles([1,2,3],[8,6,space],[7,5,4]), tiles([1,2,3],[8,space,4],[7,6,5]), Path). Path = [tiles([1, 2, 3], [8, 6, space], [7, 5, 4]), tiles([1, 2, 3], [8, 6, 4], [7, 5, space]), tiles([1, 2, 3], [8, 6, 4], [7, space, 5]), tiles([1, 2, 3], [8, space, 4], [7, 6, 5])] Yes it works!
Breadth first search w/ cycle checking %%% bfs.pro Dave Reed 2/15/02 %%% Breadth first search with cycle checking %%% bfs(Current, Goal, Path): Path is a list of states %%% that lead from Current to Goal with no duplicate states. %%% %%% bfs_help(ListOfPaths, Goal, Path): Path is a list %%% of states that lead from one of the paths in ListOfPaths %%% (a list of lists) to Goal with no duplicate states. %%% %%% extend(Path, ListOfPaths): ListOfPaths is the list of all %%% possible paths obtainable by extending Path (at the head) %%% with no duplicate states. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% bfs(State, Goal, Path) :- bfs_help([[State]], Goal, RevPath), reverse(RevPath, Path). bfs_help([[Goal|Path]|_], Goal, [Goal|Path]). bfs_help([Path|RestPaths], Goal, SolnPath) :- extend(Path, NewPaths), append(RestPaths, NewPaths, TotalPaths), bfs_help(TotalPaths, Goal, SolnPath). extend([State|Path], NewPaths) :- bagof([NextState,State|Path], (move(State, NextState), not(member(NextState, [State|Path]))), NewPaths), !. extend(_, []). since you are already having to store entire paths for BFS, cycle checking is easy when extending, exclude states already on the path
Examples w/ cycle checking ?- bfs(loc(omaha), loc(los_angeles), Path). Path = [loc(omaha), loc(chicago), loc(los_angeles)] ; Path = [loc(omaha), loc(denver), loc(los_angeles)] ; Path = [loc(omaha), loc(chicago), loc(denver), loc(los_angeles)] ; No finds all paths without cycles, shorter paths first ?- bfs(jugs(0,0), jugs(2,0), Path). Path = [jugs(0, 0), jugs(0, 3), jugs(3, 0), jugs(3, 3), jugs(4, 2), jugs(0, 2), jugs(2, 0)] ; Path = [jugs(0, 0), jugs(4, 0), jugs(1, 3), jugs(1, 0), jugs(0, 1), jugs(4, 1), jugs(2, 3), jugs(2, 0)] ; Path = [jugs(0, 0), jugs(4, 0), jugs(4, 3), jugs(0, 3), jugs(3, 0), jugs(3, 3), jugs(4, 2), jugs(0, 2), jugs(..., ...)] Yes finds 14 unique paths, shorter paths first ?- bfs(tiles([1,2,3],[8,6,space],[7,5,4]), tiles([1,2,3],[8,space,4],[7,6,5]), Path). Path = [tiles([1, 2, 3], [8, 6, space], [7, 5, 4]), tiles([1, 2, 3], [8, 6, 4], [7, 5, space]), tiles([1, 2, 3], [8, 6, 4], [7, space, 5]), tiles([1, 2, 3], [8, space, 4], [7, 6, 5])] Yes same as before (by definition, shortest path will not have cycle)
DFS vs. BFS • Advantages of DFS • requires less memory than BFS since only need to remember the current path • if lucky, can find a solution without examining much of the state space • with cycle-checking, looping can be avoided • Advantages of BFS • guaranteed to find a solution if one exists – in addition, finds optimal (shortest) solution first • will not get lost in a blind alley (i.e., does not require backtracking or cycle checking) • can add cycle checking with very little cost note: just because BFS finds the optimal solution, it does not necessarily mean that it is the optimal control strategy!
Iterative deepening • interesting hybrid: DFS with iterative deepening • alter DFS to have a depth bound • (i.e., search will fail if it extends beyond a specific depth bound) • repeatedly call DFS with increasing depth bounds until a solution is found • DFS with bound = 1 • DFS with bound = 2 • DFS with bound = 3 • . . . • advantages: • yields the same solutions, in the same order, as BFS • doesn't have the extensive memory requirements of BFS • disadvantage: • lots of redundant search – each time the bound is increased, the entire search from the previous bound is redone!
Depth first search with iterative deepening %%% dfsdeep.pro Dave Reed 2/15/02 %%% Depth first search with iterative deepening %%% dfs_deep(Current, Goal, Path): Path is a list of states %%% that lead from Current to Goal. %%% %%% dfs_deep_help(State, Goal, Path, DepthBound): Path is a %%% list of states (length = DepthBound) that lead from %%% State to Goal. THIS PREDICATE PERFORMS THE ITERATIVE %%% DEEPENING BY INCREASING THE DEPTH BOUND BY 1 EACH TIME. %%% %%% dfs_bounded(State, Goal, Path, DepthBound): Path is a %%% list of states (length = DepthBound) that lead from %%% State to Goal. THIS PREDICATE PERFORMS THE BOUNDED DFS, %%% REQUIRING A SOLUTION THAT IS EXACTLY DepthBound LONG. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% dfs_deep(State, Goal, Path) :- dfs_deep_help(State, Goal, Path, 1). dfs_deep_help(State, Goal, Path, DepthBound) :- dfs_bounded(State, Goal, Path, DepthBound). dfs_deep_help(State, Goal, Path, DepthBound) :- NewDepth is DepthBound + 1, dfs_deep_help(State, Goal, Path, NewDepth). dfs_bounded(Goal, Goal, [Goal], 1). dfs_bounded(State, Goal, [State|Path], DepthBound) :- DepthBound > 0, move(State, NextState), NewDepth is DepthBound - 1, dfs_bounded(NextState, Goal, Path, NewDepth). can even set up the bounds checking so that answers are not revisited on each pass – only report answer first time it is found
DFS w/ iterative deepening & cycle checking %%% dfsdeep.pro Dave Reed 2/15/02 %%% Depth first search with iterative deepening & cycle checking %%% dfs_deep(Current, Goal, Path): Path is a list of states %%% that lead from Current to Goal, with no duplicate states. %%% %%% dfs_deep_help(State, Goal, Path, DepthBound): Path is a %%% list of states (length = DepthBound) that lead from %%% State to Goal, with no duplicate states. THIS PREDICATE %%% PERFORMS THE ITERATIVE DEEPENING BY INCREASING THE DEPTH %%% BOUND BY 1 EACH TIME. %%% %%% dfs_bounded(State, Goal, Path, DepthBound): Path is a %%% list of states (length = DepthBound) that lead from %%% State to Goal with no duplicate states. THIS PREDICATE %%% PERFORMS THE BOUNDED DFS, REQUIRING A SOLUTION THAT IS %%% EXACTLY DepthBound LONG. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% dfs_no_cycles_deep(State, Goal, Path) :- dfs_deep_help([State], Goal, RevPath, 1), reverse(RevPath, Path). dfs_deep_help(PathSoFar, Goal, Path, DepthBound) :- dfs_bounded(PathSoFar, Goal, Path, DepthBound). dfs_deep_help(PathSoFar, Goal, Path, DepthBound) :- NewDepth is DepthBound + 1, dfs_deep_help(PathSoFar, Goal, Path, NewDepth). dfs_bounded([Goal|PathSoFar], Goal, [Goal|PathSoFar], 1). dfs_bounded([State|PathSoFar], Goal, Path, DepthBound) :- DepthBound > 0, move(State, NextState), not(member(NextState, [State|PathSoFar])) , NewDepth is DepthBound - 1, dfs_bounded([NextState,State|PathSoFar], Goal, Path, NewDepth). as with DFS, can add cycle checking by storing the current path and checking for duplicate states
Examples w/ cycle checking ?- dfs_no_cycles_deep(loc(omaha), loc(los_angeles), Path). Path = [loc(omaha), loc(chicago), loc(los_angeles)] ; Path = [loc(omaha), loc(denver), loc(los_angeles)] ; Path = [loc(omaha), loc(chicago), loc(denver), loc(los_angeles)] ; finds same answers as BFS, but hangs after last answer -- WHY? ?- dfs_no_cycles_deep(jugs(0,0), jugs(2,0), Path). Path = [jugs(0, 0), jugs(0, 3), jugs(3, 0), jugs(3, 3), jugs(4, 2), jugs(0, 2), jugs(2, 0)] ; Path = [jugs(0, 0), jugs(4, 0), jugs(1, 3), jugs(1, 0), jugs(0, 1), jugs(4, 1), jugs(2, 3), jugs(2, 0)] ; Path = [jugs(0, 0), jugs(4, 0), jugs(4, 3), jugs(0, 3), jugs(3, 0), jugs(3, 3), jugs(4, 2), jugs(0, 2), jugs(..., ...)] Yes finds same answers as BFS, but similarly hangs ?- bfs(tiles([1,2,3],[8,6,space],[7,5,4]), tiles([1,2,3],[8,space,4],[7,6,5]), Path). Path = [tiles([1, 2, 3], [8, 6, space], [7, 5, 4]), tiles([1, 2, 3], [8, 6, 4], [7, 5, space]), tiles([1, 2, 3], [8, 6, 4], [7, space, 5]), tiles([1, 2, 3], [8, space, 4], [7, 6, 5])] Yes finds same answers as BFS, noticeably faster
Cost of iterative deepening • just how costly is iterative deepening? 1st pass searches entire tree up to depth 1 2nd pass searches entire tree up to depth 2 (including depth 1) 3rd pass searches entire tree up to depth 3 (including depths 1 & 2) • in reality, this duplication is not very costly at all! • Theorem: Given a "full" tree with constant branching factor B, the number of nodes at any given level is greater than the number of nodes in all levels above. • e.g., Binary tree (B = 2): 1 2 4 8 16 32 64 128 … • repeating the search of all levels above for each pass merely doubles the amount of work for each pass. • in general, this repetition only increases the cost by a constant factor < (B+1)/(B-1)