770 likes | 867 Views
Graph Games with Reachabillity Objectives: Mixing Chess, Soccer and Poker . Krishnendu Chatterjee 5 th Workshop on Reachability Problems, Genova , Sept 30, 2011 . TexPoint fonts used in EMF.
E N D
Graph Games with Reachabillity Objectives: Mixing Chess, Soccer and Poker KrishnenduChatterjee 5th Workshop on Reachability Problems, Genova, Sept 30, 2011 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAA
Games on Graphs • Games on graphs. • History • Zermelo’s theorem about Chess in 1913 • From every configuration • Either player 1 can enforce a win. • Or player 2 can enforce a win. • Or both players can enforce a draw.
Chess: Games on Graph • Chess is a game on graph. • Configuration graph.
Graphs vs. Games Two interacting players in games: Player 1 (Box) vs Player 2 (Diamond).
Game Graphs • A game graph G= ((S,E), (S1, S2)) • Player 1 states (or vertices) S1 and similarly player 2 states S2, and (S1, S2) partitions S. • E is the set of edges. • E(s) out-going edges from s, and assume E(s) non-empty for all s. • Game played by moving tokens: when player 1 state, then player 1 chooses the out-going edge, and if player 2 state, player 2 chooses the outgoing edge.
Strategies • Strategies are recipe how to move tokens or how to extend plays. Formally, given a history of play (or finite sequence of states), it chooses a probability distribution over out-going edges. • ¾: S*S1 D(S). • ¼: S*S2! D(S).
Strategies • Strategies are recipe how to move tokens or how to extend plays. Formally, given a history of play (or finite sequence of states), it chooses a probability distribution over out-going edges. • ¾: S*S1! D(S). • History dependent and randomized. • History independent: depends only current state (memoryless or positional). • ¾: S1! D(S) • Deterministic: no randomization (pure strategies). • ¾: S*S1! S • Deterministic and memoryless: no memory and no randomization (pure and memoryless and is the simplest class). • ¾: S1! S • Same notations for player 2 strategies ¼.
Objectives • Objectives are subsets of infinite paths, i.e., õ S!. • Reachability: there is a set of good vertices (example check-mate) and goal is to reach them. Formally, for a set T if vertices or states, the objective is the set of paths that visit the target T at least once.
Applications: Verification and Control of Systems • Verification and control of systems • Environment • Controller M satisfies property (Ã) E C
Applications: Verification and Control of Systems • Verification and control of systems • Question: does there exists a controller that against all environment ensures the property. C M E satisfies property (Ã) || ||
Game Models Applications -synthesis [Church, Ramadge/Wonham, Pnueli/Rosner] -model checking of open systems -receptiveness [Dill, Abadi/Lamport] -semantics of interaction [Abramsky] -non-emptiness of tree automata [Rabin, Gurevich/ Harrington] -behavioral type systems and interface automata[deAlfaro/ Henzinger] -model-based testing [Gurevich/Veanes et al.] -etc.
Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. X T
Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. X T
Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. X T
Reachability Games • Pre(X): given a set X of states, Pre(X) is the set of states such that player 1 can ensure next state in X. • Fix-point X T
Reachability Games • Winning set for a partition: Determinacy • Player 1 wins: then no matter what player 2 does, certainly reach the target. • Player 2 wins: then no matter what player 1 does, the target is never reached. • Memoryless winning strategies. • Can be computed in linear time [Beeri 81, Immerman 81].
Chess Theorem • Zermelo’s Theorem Win1 Win2 Both draw
Game Graphs Till Now • Game graphs we have seen till now • Many rounds (possibly infinite). • Turn-based.
Simultaneous Games • Theory of rational behavior as game theory • von Neumann- Morgenstern games • Matrix zero-sum games R P S R (0,0) (-1,1) (1,-1) P (1,-1) (0,0) (-1,1) S (-1,1) (1,-1) (0,0)
Simultaneous Games • Theory of rational behavior as game theory • von Neumann- Morgenstern games • Matrix zero-sum games R P S R (0,0) (-1,1) (1,-1) P (1,-1) (0,0) (-1,1) S (-1,1) (1,-1) (0,0)
Simultaneous Games • Example: Prisoners dilemma. • Another example. R L C R (1,-1) (-1,1) (-1,1) L (-1,1) (1,-1) (-1,1) C (-1,1) (-1,1) (1,-1)
Simultaneous Games • Example: Prisoners dilemma. • Another example. R L C R (1,-1) (-1,1) (-1,1) L (-1,1) (1,-1) (-1,1) C (-1,1) (-1,1) (1,-1)
Simultaneous Games • Another example: Penalty shoot-out (Soccer) R L C R (1,-1) (-1,1) (-1,1) L (-1,1) (1,-1) (-1,1) C (-1,1) (-1,1) (1,-1)
Chess Vs. Soccer (Penalty) • Chess: • Turn-based • Possibly infinite rounds • Theory of simultaneous games (Soccer) • Concurrent • One-shot (one-round) • Mix chess and soccer • Concurrent games on graphs
Concurrent Game Graphs • A concurrent game graph is a tuple G =(S,M,¡1,¡2,±) • S is a finite set of states. • M is a finite set of moves or actions. • ¡i: S !2Mn; is an action assignment function that assigns the non-empty set ¡i(s) of actions to player i at s, where i2 {1,2}. • ±: S £ M £ M ! S, is a transition function that given a state and actions of both players gives the next state.
An Example: Snow-ball Game Hide Run Throw Wait run, wait hide, throw [Everett 57] run, throw R s hide, wait
New Solution Concepts • Sure winning for turn-based. • New solution concepts • Almost-sure winning. • Limit-sure winning.
Almost-sure Winning Example head, head tail, tail R s head, tail tail, head Almost-sure winning strategy: say head and tail with probability ½. Randomization is necessary.
Concurrent reachabilitygames: limit-sure Hide Run Throw Wait run, wait hide, throw [Everett 57] run, throw R s hide, wait MoveProbability runq hide1-q (q>0) Win at s with probability 1-q, for all q> 0.
Concurrent reachability games: limit-sure Hide MoveProbability runq hide1-q (q>0) Run Throw Wait Win at s with probability 1-q, for all q > 0. w = 0 1 1 run, wait hide, throw [Everett 57] run, throw R s hide, wait Player 1 cannot achieve w(s) = 1, only w(s) = 1-qfor all q > 0.
Results for Concurrent Reachability Games • Sure winning: • Deterministic memoryless sufficient. • Linear time. • Almost-sure winning: • Randomization is necessary. • Randomized memoryless is sufficient. • Quadratic time algorithm. • Limit-sure winning: • Randomization is necessary. • Randomized memoryless is sufficient. • Quadratic time algorithm. • Results from [dAHK98, CdAH06, CdAH09]
Games Till Now • Turn-based graph games • Concurrent graph games • Applications: again verification and synthesis with synchronous interaction. • Both these games are perfect-information games. Players know the precise state of the game. • The game of Poker: players play but do not know the perfect state of the game.
Why Partial-information • Perfect-information: controller knows everything about the system. • This is often unrealistic in the design of reactive systems because • systems have internal state not visible to controller (private variables) • noisy sensors entail uncertainties on the state of the game • Partial-observation Hidden variables = imperfect information. Sensor uncertainty = imperfect information.
Partial-information Games • A PIG G =(L, A, , O) is as follows • L is a finite set of locations (or states). • A is a finite set of input letters (or actions). • µ L £ A £ L non-deterministic transition relation that for a state and an action gives the possible next states. • O is the set of observations and is a partition of the state space. The observation represents what is observable. • Perfect-information: O={{l} | l 2 L}.
PIG: Example b a a,b b a
New Solution Concepts • Sure winning: winning with certainty (in perfect information setting determinacy). • Almost-sure winning: win with probability 1. • Limit-sure winning: win with probability arbitrary close to 1. • We will illustrate the solution concepts with card games.
Card Game 1 • Step 1: Player 2 selects a card from the deck of 52 cards and moves it from the deck (player 1 does not know the card). • Step 2: • Step 2 a: Player 2 shuffles the deck. • Step 2 b: Player 1 selects a card and view it. • Step 2 c: Player 1 makes a guess of the secret card or goes back to Step 2 a. • Player 1 wins if the guess is correct.
Card Game 1 • Player 1 can win with probability 1: goes back to Step 2 a until all 51 cards are seen. • Player 1 cannot win with certainty: there are cases (though with probability 0) such that all cards are not seen. Then player 1 either never makes a guess or makes a wrong guess with positive probability.
Card Game 2 • Step 1: Player 2 selects a new card from an exactly same deck and puts is in the deck of 52 cards (player 1 does not know the new card). So the deck has 53 cards with one duplicate. • Step 2: • Step 2 a: Player 2 shuffles the deck. • Step 2 b: Player 1 selects a card and view it. • Step 2 c: Player 1 makes a guess of the secret duplicate card or goes back to Step 2 a. • Player 1 wins if the guess is correct.
Card Game 2 • Player 1 can win with probability arbitrary close to 1: goes back to Step 2 a for a long time and then choose the card with highest frequency. • Player 1 cannot win probability 1, there is a tiny chance that not the duplicate card has the highest frequency, but can win with probability arbitrary close to 1, (i.e., for all ² >0, player 1 can win with probability 1- ², in other words the limit is 1).
Sure winning for Reachability • Result from [Reif 79] • Memory is required. • Exponential memory required. • Subset construction: what subsets of states player 1 can be. Reduction to exponential size turn-based games. • EXPTIME-complete.
Almost-sure winning for Reachability • Result from [CDHR 06, CHDR 07] • Standard subset construction fails: as it captures only sure winning, and not same as almost-sure winning. • More involved subset construction is required. • EXPTIME-complete.