Evolving New Strategies

1. Evolving New Strategies The Evolution of Strategies in the Iterated Prisoner’s Dilemma 01 / 25

2. What is the Prisoner’s Dilemma? • There are two prisoners • Each one has taken part in the same criminal act • The authorities are interrogating each one • Each prisoner can choose to keep their mouth shut or rat out their partner • If both prisoners stay quiet, they each get n months of jail time • If only one prisoner gets ratted out, that prisoner gets n + x months of jail time while the other prisoner gets n – y months of jail time • If the prisoners rat each other out, they each get n + z months of jail time. • In this case, n, x, y, and z are all greater than zero. • In this case, x is greater than z. 02 / 25

3. What is the Iterated Prisoner’s Dilemma? • Prisoner’s Dilemma performed several times • The two criminals have committed several crimes together • They are interrogated for each crime, with each set of interrogations being an instance of the original Prisoner’s Dilemma • These interrogations are performed in sequence (or iteratively), and the jail time distributed to each prisoner is cumulative 03 / 25

4. How does the IPD relate to GAs? • No optimal solution • No real strategy • No clue • Hard problem • So back to the paper 04 / 25

5. What This Paper Shows • GAs in a rich social setting • Advantage of developing new strategies • One parent • Two parent • Early commitments to paths • Evolutionary processes optimal or arbitrary 05 / 25

6. How Does It Show It? • Simulation • Multiple cases • Comparative output 06 / 25

7. The Simulation • Specify the environment • Specify the encoding • Testing the effects of random mutation • Run the simulation • Analyze the results 07 / 25

8. The Environment • Prisoner’s dilemma • Multiple prisoners • Goal is to achieve mutual cooperation • Individuals may meet more than once 08 / 25

9. Initial Experiment • Original strategies were submitted by fourteen people • Game Theory • Economics • Sociology • Political Science • Mathematics • Various levels of intricacy 09 / 25

10. Initial Experiment • Most complex strategy • Markov process • Bayesian inference • Least complex strategy • TFT • TFT won 10 / 25

11. Second Experiment • Sixty-two entries • Six countries • Computer hobbyists, professors • TFT was submitted again • It won 11 / 25

12. The GA • Population • Encoding • Generation • Crossover • Mutation • Fifty Generations 12 / 25

13. Population • Twenty chromosomes • Seventy genes 13 / 25

14. Encoding • For each prisoner’s dilemma, there are four possibilities • Each “player” has memory • What each gene represents 14 / 25

15. A Single Generation • Multiple games • Each game had one-hundred and fifty-one moves • Each chromosome played eight others • Fitness was assigned • Ratted out – Zero points • Mutually ratted out – One point • Mutual cooperation – Three points • You ratted, other person stayed quit – Five points 15 / 25

16. Crossover • Fitness proportional selection • Involved standard deviation from mean • Strictly ten crossovers • Single point • Two parents 16 / 25

17. Mutation • Single gene flip • One gene per two chromosomes 17 / 25

18. Results • Median resultant member • Just as good as TFT • Resembled TFT • Five properties were found • Don’t rock the boat • Be provocable • Accept apologies • Forget • Accept a rut 18 / 25

19. Results • ADJUSTER • Special chromosome which consistently seeks to exploit • TFT • Majority of other chromosomes 19 / 25

20. Results • Twenty-five percent of runs • Median was better • Exploit one chromosome 20 / 25

21. Results: Why is this important? • Chromosomes had to learn • Discriminatory based on evidence • Self adjusting for exploitation • No alienation • Break primary rule of first tournament • Be nice? I don’t think so 21 / 25

22. Results: Misleading? • Median • Exploitative • Menacing • A true criminal? • Fixed size population and tournaments • Simulate real evolution 22 / 25

23. Results: A Slight Twist • Asexual reproduction • TFT • Less than half of the medians • Changing environment • Play against everyone • Everyone starts aggressive • Fitness rapidly declines • Fitness begins to even out • Fitness begins to rise 23 / 25

24. Conclusions • The GA is good for searching, large, multi-dimensional spaces • Multiple parent crossover helps • Arbitrary aspects of evolution • Hitch hikers • Exploration vs. Exploitation • Selection Pressure • Evolutionary Commitments can be irreversible 24 / 25

25. Related Topics • Mutation • Crossover • Inversion • Coding principles • Dominant/Recessive • Rate of evolution • Population viscosity • Speciation and niches 25 / 25