Finite State Machine for Games

1. Finite State Machine for Games Spring 2005 Ref: Chenney, CS679 lectures AI Game Programming Wisdom 2

2. Outline • AI and Game • Introduction/examples • Design • Intuition • State-based • Implementation • Extending • Stack-based • Fuzzy-state machine

3. What is AI? • AI is the control of every non-human entity in a game • The other cars in a car game • The opponents and monsters in a shooter • Your units, your enemy’s units and your enemy in a RTS game • But, typically does not refer to passive things that just react to the player and never initiate action • That’s physics or game logic • For example, the blocks in Tetris are not AI, nor is a flag blowing in the wind

4. AI in the Game Loop • AI is updated as part of the game loop, after user input, and before rendering • There are issues here: • Which AI goes first? • Does the AI run on every frame? (LOD problem) • Is the AI synchronized?

5. AI and Animation • AI determines what to do and the animation does it • AI drives animation, deciding what action the animation system should be animating • Scenario 1: The AI issues orders like “move from A to B”, and it’s up to the animation system to do the rest • Scenario 2: The AI controls everything down to the animation clip to play • Which scenario is best depends on the nature of the AI system and the nature of the animation system • Is the animation system based on move trees (motion capture), or physics, or something else • Does the AI look after collision avoidance? Does it do detailed planning?

6. AI Update Step • The sensing phase determines the state of the world • May be very simple - state changes all come by message • Or complex - figure out what is visible, where your team is, etc • The thinking phase decides what to do given the world • The core of AI • The acting phase tells the animation what to do • Generally not interesting AI Module Sensing Game Engine Thinking Acting

7. AI by Polling • The AI gets called at a fixed rate • Senses: It looks to see what has changed in the world. For instance: • Queries what it can see • Checks to see if its animation has finished running • And then acts on it • Why is this generally inefficient? • Different characters might require different polling rate

8. Event Driven AI • Event driven AI does everything in response to events in the world • Events sent by message (basically, a function gets called when a message arrives, just like a user interface) • Example messages: • A certain amount of time has passed, so update yourself • You have heard a sound • Someone has entered your field of view • Note that messages can completely replace sensing, but typically do not. Why not? • Real system are a mix - something changes, so you do some sensing

9. AI Techniques in Games • Basic problem: Given the state of the world, what should I do? • A wide range of solutions in games: • Finite state machines, Decision trees, Rule based systems, Neural networks, Fuzzy logic • A wider range of solutions in the academic world: • Complex planning systems, logic programming, genetic algorithms, Bayes-nets • Typically, too slow for games

10. Goals of Game AI • Several goals: • Goal driven - the AI decides what it should do, and then figures out how to do it • Reactive - the AI responds immediately to changes in the world • Knowledge intensive - the AI knows a lot about the world and how it behaves, and embodies knowledge in its own behavior • Characteristic - Embodies a believable, consistent character • Fast and easy development • Low CPU and memory usage • These conflict in almost every way

11. Two Measures of Complexity • Complexity of Execution • How fast does it run as more knowledge is added? • How much memory is required as more knowledge is added? • Determines the run-time cost of the AI • Complexity of Specification • How hard is it to write the code? • As more “knowledge” is added, how much more code needs to be added? • Determines the development cost, and risk

12. Expressiveness • What behaviors can easily be defined, or defined at all? • Propositional logic: • Statements about specific objects in the world – no variables • Jim is in room7, Jim has the rocket launcher, the rocket launcher does splash damage • Go to room8 if you are in room7 through door14 • Predicate Logic: • Allows general statement – using variables • All rooms have doors • All splash damage weapons can be used around corners • All rocket launchers do splash damage • Go to a room connected to the current room

13. Finite State Machines (FSMs) • A set of states that the agent can be in • Connected by transitions that are triggered by a change in the world • Normally represented as a directed graph, with the edges labeled with the transition event • Ubiquitous in computer game AI • You might have seen them, a long time ago, in formal language theory (or compilers) • What type of languages can be represented by finite state machines? • How might this impact a character’s AI? • How does it impact the size of the machine?

14. Formal Definitions (N. Philips) • "An abstract machine consisting of a set of states (including the initial state), a set of input events, a set of output events, and a state transition function. • The function takes the current state and an input event and returns the new set of output events and the next state. Some states may be designated as "terminal states". • The state machine can also be viewed as a function which maps an ordered sequence of input events into a corresponding sequence of (sets of) output events. • Finite State Automaton: the machine with no output

15. FSM with Output: vending machines • [description] • State table

16. Vending Machine: state diagram

17. FSM and Game • Game character behavior can be modeled (in most cases) as a sequence of different “mental state”, where change is driven by the actions of player/other characters, … • Natural choice for defining AI in games

18. FSM with No Output

19. Dead Ex: predator vs. prey • Prey (laalaa) Idle (stand,wave,…) Sees predator Flee (run) No more threat captured

20. Predator (Raptor) Idle (stand) Tidle > 5 Tdining>5 Hungry (wander) Dining Prey captured Prey in sight Pursuit (run) Tpursuit > 10

21. Idling LaaLaa This page illustrates: hierarchical state, Non-deterministic state transition Target arrived Wander (set random target) 20% Stand Tstand>4 30% R Wave 50% Twave>2

23. FSM + Cinematography • The current state (and state transition) may have an important effect on how camera should be manipulated • Idling: focus on character • Movement: zoom out to show spatial info • State-transition: camera animation … • More complicated camera behavior may be coded in another FSM

24. Camera Selection Strategy It is important to choose a camera that has the front of the character X Z

25. Camera Selection (cont) A good camera: -v lies in the frustum -v

26. r x w Other Concerns • Adjust zoom (fovy) so that the character occupies a good (fixed!?) portion of the screen d near

27. FOVY based on Postures • Once in running mode, zoom out • In stand/wave modes, zoom in • Smooth transition between two fovy settings (by indirectly controlling the percent value)

28. FSM Design

29. Quake2 Examples Intuitive thinking: model the events and state changes Quake2 uses 9 different states: standing, walking, running, dodging, attacking, melee, seeing the enemy, idle and searching. Incomplete design

30. Quake: Rocket

31. Shambler monster

32. Intuitive Design • Say, a simple teletube baby has three states: idle, run, and wave • Scenario: • When an idle laalaa sees a butterfly, it waves to it. When the butterfly flies away, it returns to idle • When an idle laalaa sees a mouse, it flees away. When the mouse is no longer in sight, it returns to idle

33. Laalaa flee mouse ~mouse How to make sure the design complete? I.e., all states and transitions are considered Is there any systematic way of developing an FSM? idle butterfly wave ~butterfly

34. Quake Bot Example (first-cut) • Types of behavior to capture: • Wander randomly if don’t see or hear an enemy • When see enemy, attack • When not see enemy and hear an enemy, chase enemy • When die, re-spawn (new a bot from start) • Events: see enemy, hear enemy, die • States: wander, attack, chase, spawn

35. Remark • With 3 events, potentially there should be 23 states: • (E,S,D)=(0,0,0),(1,0,0),(0,1,0), …,(1,1,1) • Some doesn’t make sense • E.g., ESD = 111 • Name and create a state for the ones that we want to consider • Wander (ESD=000) • Chase (ESD=010) • Attack (ESD=1x0), x for dont-care • Die (ESD=xx1)

36. Attack 1x0 ~E start E D Wander 000 S Chase 010 ~S D Spawn xx1 D E S D FSM (first-cut) Problem: Can’t go directly from attack to chase. Why not? • Events: • E: see an enemy • S: hear a sound • D: die • States: • E: enemy in sight • S: sound audible • D: dead

37. Attack 100 ~E start E D Wander 000 S Chase 010 ~S D Spawn xx1 D E S D FSM (first-cut) Extra state needs to be defined Attack+S 110 ~S • Events: • E: see an enemy • S: hear a sound • D: die • States: • E: enemy in sight • S: sound audible • D: dead S ~E E D

38. Quake Bot Example (refined) • Types of behavior to capture: • Wander randomly if don’t see or hear an enemy • When see enemy, attack • When not see enemy and hear an enemy, chase enemy • When die, respawn • Extensions: • When health is low and see an enemy, retreat • When see power-upsduring wandering, collect them [hierarchical FSM]

39. Example FSM with Retreat Attack-ES E,-D,S,-L Retreat-S -E,-D,S,L Attack-E E,-D,-S,-L S • States: • E: enemy in sight • S: sound audible • D: dead • L: Low health • A lot more states got added L -S L -L E -E E -L Retreat-ES E,-D,S,L Wander-L -E,-D,-S,L E -L E L -S -L L S Retreat-E E,-D,-S,L Wander -E,-D,-S,-L -E -E E D D Chase -E,-D,S,-L D D Spawn D (-E,-S,-L) S

40. Hierarchical FSMs • What if there is no simple action for a state? • Expand a state into its own FSM, which explains what to do if in that state • Some events move you around the same level in the hierarchy, some move you up a level • When entering a state, have to choose a state for its child in the hierarchy • Set a default, and always go to that • Or, random choice • Depends on the nature of the behavior

41. Hierarchical FSM Example Attack Wander ~E • Note: This is not a complete FSM • All links between top level states still exist • Need more states for wander E Chase Pick-up Powerup ~S S Spawn Start Turn Right D ~E Go-through Door

42. Approach .3 Aim & Slide Right & Shoot .3 .3 .4 .3 .4 Aim & Slide Left & Shoot Aim & Jump & Shoot Non-Deterministic HierarchicalFSM (Markov Model) • Adds variety to actions • Have multiple transitions for the same event • Label each with a probability that it will be taken • Randomly choose a transition at run-time • Markov Model: New state only depends on the previous state Attack Start

43. FSM Control System Implementation

44. FSM Implementation

45. event state Efficient Implementation • Compile into an array of state-name, event • state-namei+1 := array[state-namei, event] • Switch on state-name to call execution logic • Markov: Have array of possible transitions for every (state-name,event) pair, and choose one at random • Hierarchical • Create array for every FSM • Have stack of states • Classify events according to stack • Update state which is sensitive to current event

46. FSM Advantages • Very fast – one array access • Expressive enough for simple behaviors or characters that are intended to be “dumb” • Can be compiled into compact data structure • Dynamic memory: current state • Static memory: state diagram – array implementation • Can create tools so non-programmer can build behavior • Non-deterministic FSM can make behavior unpredictable

47. FSM Disadvantages • Number of states can grow very fast • Exponentially with number of events: s=2e • Number of arcs can grow even faster: a=s2 • Propositional representation • Difficult to put in “pick up the better powerup”, “attack the closest enemy” • Expensive to count: Wait until the third time I see enemy, then attack • Need extra events: First time seen, second time seen, and extra states to take care of counting

48. Example

49. Code 1 Ad hoc implementation

50. Code 1p