Swarm behaviour and traffic simulations

Swarm behaviour and traffic simulations Using stigmergy to solve algorithmic problems, predict and improve vehicle traffic

Overview (1) • Swarms in nature • Social insects and Stigmergy • Ant algorithms and application examples: • Foraging in ants • Using foraging behaviour to solve the TSP • Labour division among social insects • Mailmen using adaptive task allocation model

Overview (2) • Traffic simulation by cellular automata • Adopting the stigmergic process • Prediction of driver behaviour • Traffic infrastructure optimization • Drivers as ant agents • Signal lights as social insects • Other real world applications

Swarms in nature What is a swarm ???

Pictures of swarms (1)

Characteristics of swarms • Aggregation of animals with similar size and often similar orientation • Interaction of animals leads to new intelligent forms of behaviour that are not inherited in the individuals • E.g. insects, birds, fish, bacteria

The main actor of the presentation

Social insects and stigmergy • Social insect societies are distributed systems with highly structered social organization • They can accomplish complex tasks that far exceed the individuals abilities • Here: focus on stigmergy as important means of indirect communication paradigm

Stigmergy • Originally defined by Grassé: Stimulation of workers by the performance they have achieved • Method of indirect communication in a self-organizing emergent system where its individual parts communicate with each other by modifyingtheirlocal environment • Here: Pheromones  diffusing chemical substance

Stigmergy example • Example: termites buildung nest pillars with soil pellets • Stimulus  response • Autocatalytic process

Stigmergy behaviour of ants Stigmergy behaviour in ants and their transfer to computer algorithms: • Foraging and the TSP • Labour Division and adaptive task allocation

Foraging in ants • Foraging means searching for food • Ants manage to find the shortest path between their nest and a food source • Achieved through trail-laying and trail–following behaviour with pheromones  Stigmergy

Foraging example • Two paths with different lengths • Ants follow way with most pheromones • Autocatalytic process leads to „differential length effect“

The Travelling Salesman Problem • Consists of a set of given cities • Goal is to visit all cities in a closed loop of shortest length • Every city must be visited only once! • E.g. 15 biggest cities of Germany

TSP represented by graph theory TSP defined more generally by graph theory: • Graphs consist of vertices V and edges E • Cities are vertices, edges are connections between cities • In the TSP each city is connected to each other! • Each edge has a certain length • Example: 4 cities A,B,C,D represented by vertices; 6 connnections with lengths represented by edges

Ants solving the TSP • Artificial ants exploring the TSP graph • Artificial pheromones added by ants after completion of a complete loop proportional to 1/length of route • Probabilistic transition rule for ant k to next city j: • City j visited? • Length of edge gives desirability measure ηij = 1/length • Amount of pheromones τij(t) on edge (i, j) • Evaporation of pheromones over time lets system forget bad information

Labour division in ants • Fundamental in social insects: Division of reproductive castes from worker castes • Further divisions: • subcastes of workers  specialists • subcastes of age and morphology • again dividing subcastes into behavioural castes • Plasticity: Workers switch tasks in response to internal and external pertubations

Labour division model • Based on idea of response threshold • Stimulus exceeds individuals response threshold  individual engages in task performance • Stimulus plays role of Stigmergy here (can be pheromones, amount of encounters, …)

Extended labour division model More realistic: Extend previous model by threshold varying in time.  if an individual performs a certain task its threshold related to this task decreases  the thresholds related to all other tasks, not performed meanwhile, are increasing

Example: Express mail retrieval (1) • Group of mailmen has to pick up letters in a city • Goal: Allocation of mailmen to appearing demands should be optimal  realized with adaptive task allocation model • Each mailmen i reacts with a certain probability p to arising demands, depending on: • response threshold Ө related to area j with demand • the distance d to the area with demand • the intensity s of the demand  stimulus

Example: Express mail retrieval (2) • Figure (a): demand of a certain area over time • At t = 2000 the mailman that is specialized on this area gets removed • Figure (b): response threshold of another mailmen that is reacting to the loss of the specialist

Traffic simulations Why would one do that? • to predict drivers behaviour in order to adjust dynamic traffic signs, or propose alternative routes in navigation devices or radio • to improve traffic infrastructureand traffic light plans in big, complicated traffic networks like cities

Cellular automata traffic models (1) • Two major approaches on traffic simulation: • Fluid-dynamical, with continuous traffic  macroscopic • Discretized cellular automata model  microscopic • Focus in presentation: discretized cellular automata models • Discretized: • Street is diveded into fixed sites, cars have integer velocities • Each site can be occupied by a car or can be empty x meters car 1 car 2 e.g. one lane traffic site n site n+1 site n +2 site n +3 site n + 4

Cellular automata traffic models (2) Assuming a simple one lane model: • One update of the system consists of the following steps performed with each car in parallel: • Acceleration or slowing down: Depending on maximal speed and distance to next car • Randomization: To contribute human behaviour and external influences • Car motion: The advancment of sites, according to the speed • Model shows nontrivial and realistic behaviour

Adopting Stigmergy • Cars adopt the pheromone laying and sniffing behaviour of ants • Leads to very realistic and dynamic system • Reduces communication between cars to local information creation and retrieval  stigmergy • Computational costs can be reduced for collision checking • Still, information about traffic signs and other environmental signals are non-local  „cellular automata ant cars“

How cars behave like ants • Each car leaves and sniffs pheromones on the road • Pheromones fade over time, like with ants • Faster cars leave longer trails then slower cars (a) • Additional pheromone dropping necessary for • stopped cars (b) • quick deceleration • lane changing (c) • like using signal and brake lights (a) (b) (c)

Traffic prediction • Measurement devices like cameras determine the vehicles entering an area • Implementing foraging behaviour of ants leads to realistic system of interacting drivers  drivers follow other drivers and try to escape jams • Various types of driver support: • Adjustment of dynamic traffic signs to avoid congestions • Knowledge of growth of traffic jams allows to give reasonable redirections • Through use of foraging behaviour alternative routes can be given more effectively, cars spread more

Optimizing traffic light plans • Microscopic traffic model by individual cars with individual aims • 1st Approach: • Cars as agents facilitate change of light plans by voting • Evolutionary process improves overall light plans • 2nd Approach: • Groups of ligths at an intersection behave like social insects • Adaptive task allocation is responsible for running plans

Cars voting for traffic ligths • Each car keeps track of two variables: • total driving time dtot • total waiting time wtot  waiting measure: • Statistics give information about overall fitness  overall waiting measure: • Cars that are stopped at a light vote for it • Lights with many votes are more probable to be mutated  quicker adaption in the evolutionary process

Evolutionary process • Probabilistic mutations of traffic light timings • Mutation changes length of correlated light phases at an intersection (e.g. N–S and E-W) • Simulation of each branch of a new generation • Survival of the fittest  with least waiting time of cars simulation 1 mutate mutate choose fittest simulation 2 . . . . simulation 3

Traffic Simulation under SuRJE SuRJE: Swarms Under R&J using Evolution • Design environment to build, test and optimize traffic scenarios • Uses swarm based approach and evolutionary adaption • Features: • Enables to build multi-lane road maps • Car seeding areas define the input and output of cars • Initial lights settings for starting point • Evolutionary adaption parameters can be set

Simulation example in SuRJE • Example network: „Looptown“ (a) • Figure (b) shows the decrease of overall waiting time over generations

Traffic lights as social insects • Traffic lights are implemented as social insects  all lights at an intersection form one insect • Insect has to perform one traffic light plan out of several for its intersection • Traffic can be modeled by any microscopic traffic simulator • Cars emit pheromones when crossing and waiting at an intersection to provide stimulus

Use of adaptive task allocation • Adaptive task allocation model is applied to insects • light-plans are chosen through stimulus – response strategies • communication of intersections only by Stigmergy • Each insect/intersection has individual thresholds related to available light-plan • Stimulus for light-plan j provided by cars: • Reinforcement learning is used to specialize intersections: • Threshold j of intersection i: • Learning coefficient:

Example of stimulus evaluation • 4 way intersection with two light-plans: • 1st plan gives priority to North-South leading lanes • 2nd plan has priority for West-East leading lanes • Traffic situation: • Many cars are driving on West – East direction • Few cars on North-South direction • Initially plan 1 is driven: • Big amount of pheromones for W-E (many, waiting cars) • Small amount for N-S (few, almost not waiting cars)  evaluation of stimulus yields higher value for 2nd plan! N W E S

Traffic simulation - Recapitulation • Traffic prediction: • Simulation into future by using swarm based approach • Giving the driver useful information about choosing the best route • Traffic light timing improvment by cars as agents: • Cars are modeled with swarm based approach • Improvement of traffic light plans through evolution • Good for simple traffic lights with static timing program • Dynamic traffic light adoption through social insects: • Cars are modeled by any microscopic traffic simulation • Intersections choose their light plan through adaptive task allocation • Good for traffic lights with sensors for counting cars

Examples for real world applications • Scheduling Problems, e.g. subway, train • Vehicle Routing, e.g. bus, taxi • Connection-oriented network routing, e.g. internet, TCP/IP • Connection-less network routing, e.g. bluetooth, infrared • Optical networks routing

Thank you for your attention!

Swarm behaviour and traffic simulations