Parallelized Evolution System

Parallelized Evolution System Dept. of Computer Engineering Middle East Technical University http://www.kovan.ceng.metu.edu.tr Onur Soysal, Erkin Bahçeci Erol Şahin ISCIS’03 04/11/2003

........... Chrom.1: 1010001110... Chrom.2: 0011110101... ......... Population n Generation n SelectReproduceMutate Chrom.1: 0101011001... Chrom.2: 1100110111... ......... Generation n+1 Population n+1 ........... Evolutionary Methods • Simulation of Natural Evolution • Using evolution mechanisms to solve optimization problems • Survival of the fittest

Genetic Algorithms • Encoding • Binary representation • Stochastic search • Guided by measure of fitness • Problem-specific fitness function • Genetic operators • Mutation • Crossover

Controller ??? Sensors Actuators Evolutionary Robotics • Motivation • Controller: Sensor-actuator relationship • Difficult to design manually in nontrivial tasks • To be in the shoes of a robot? • Solutions of evolutionary robotics • Evolving controllers

Controller Genotype  Phenotype:Generic Controller Sensor data Chromosome Controller 010101 100111... Convert to controller parameters Use the controller in robots Actuator outputs

Genotype  Phenotype:Neural Network Controller Sensor data Chromosome 010101 100111... Convert to neural network weights Use the neural network to control robots Actuator outputs

Fitness Evaluation • Use controller • on physical robots • on simulated robots • Observe behavior • Calculate fitness value • Repeat • for each generation • for each chromosome

Physics Based Simulation • Importance of realistic simulations • Pros • Much faster runtime compared to physical implementation • Behavioral similarity to physical robots • Realistic interactions • Cons • High processing demand! • Example libraries: ODE, Vortex, Havok

Single Machine Limitations • Computation required • Solving Ordinary Differential Equations • Increasing complexity with more collisions • Time estimates for single computer • Order of minutes for a single evaluation • For 100 chromosomes and 100 generations • Total time > a week on a single machine

Cluster Computing • Cheap alternative to supercomputers • Large-grained problems • Example application • SETI@home: Search for Extraterrestrial Intelligence at home

Cluster Computing Solution to Evolutionary Robotics • Fitness evaluations dominate computation time • Fitness evaluations are independent • Master-Slave model • Master: GA • Slaves: Fitness evaluations

Parallel Evolution System (PES) • Parallel Genetic Algorithm library • Distributed fitness evaluation • Central genetic algorithm • Supports interoperability of multiple platforms

PES-C Client Application Server Application PES-S PES-C Client Application PES-C Client Application PES Architecture • Server: GA • Clients: Fitness evaluation

ping PES-S PES-C ping reply PES Network Adapter PES Network Adapter task result PES Communication Model • Ping mechanism • Task and result packets

PES-S Architecture PES-S Network Adapter Task generator Server Application Task Manager Best solutions GA Engine

PES-C Task Client Application Network Adapter Fitness PES-C Architecture

PES Usage: Server • Server Code #include <pes.h>int main(){ PES_INIT(); for each generation { PES_EVAL_INDIVIDUALS(); PES_NEXT_GENERATION(); }}

PES Usage: Client • Client Code #include <pes.h>int main(){ PES_INIT(); while true { RECEIVE_TASK(); EVAL_TASK(); SEND_RESULT(); }}

PES Features • Multi-platform support • Linux and Windows via PVM • Improved fault tolerance • Ping mechanism to detect non-responding clients • Supplies GA facilities • Built-in crossover, mutation and selection functions • Simplified usage • Automatic task distribution / fitness collection • Single line for evaluating fitness of whole population • Basic load balancing

Our Application • Task: Aggregation • Getting 10 robots as close as possible • Minimal sensors and communication • Modified single layer NN for control • No memory/learning • No world knowledge • Probabilistic inputs • Fitness evaluation • 2000 steps of simulation • Total distance as fitness measure 1 2 1 2 3 3 1 2 3 R 1 2 3

Sample Application Results

Processor Load Balancing • Dynamic simulation • Varying number of collisions • Varying task complexity • Varying processor load Diamonds: tasks Solid lines: Start of new generation

Fault Tolerance • Processor 2 fails • Detected at ping at 15th sec • Task restart at 19th sec Red lines: PingBlue lines: GenerationNumbers: Task index

Efficiency & Speedup

Discussions & Future Work • Generation gap • Asynchronous reproduction • Dynamic host list • Progress monitor • Extension to the Internet

PES Sources & Documentation • Source distribution under Gnu Public License (GPL) • User manual • Functional reference • Sample applications http://www.kovan.ceng.metu.edu.tr/software/PES

Parallelized Evolution System