62. The Evolution of Cooperation (part 2 of 2)

Note: This post is the sixty-second in a series about government and commercial ethics. Click here for the full listing of the series. The first post in the series has more detail on the book 'Systems of Survival' by Jane Jacobs which inspired this series.

This post is a continuation of last week's post which began discussing The Evolution of Cooperation by Robert Axelrod and covered the first 6 chapters.

In chapter 7 Axelrod provides advice for how people should act if they are in a position where, rather than accepting that they are in a Prisoner's Dilemma and trying to choose the best action based on their situation, they can try to change the situation.

Axelrod notes that while in most cases people will want to influence the situation to encourage cooperation, there are some cases (such as when businesses collude, or in the original Prisoner's Dilemma when the interrogator was trying to get the Prisoners to rat on each other) where the goal will be to discourage cooperation.

So the following advice is for encouraging cooperation in the Prisoner's Dilemma - if you want to discourage cooperation, do the opposite.

1. Enlarge the shadow of the future.

What this means is that interactions between people should be structured so that the same people meet repeatedly rather than meeting a different person every time. It might also mean setting things up so that interactions are more frequent and spaced closer together in time. Basically any change that increases the potential risk of suffering retaliation if you choose to defect rather than cooperate with someone.

2. Change the payoffs

The bigger the payoff for defecting, the greater the temptation to do so. So reducing the payoff from defection or increasing the payoff from cooperation makes cooperation more likely. For example, the potential of an audit, helps encourage people to do their taxes accurately

3. Teach people to care about each other

This is really just another way to change the payoffs. If you care for the other person's wellbeing as much as your own, than their can never really be a Prisoner's Dilemma. And every increase in empathy reduces the element of dilemma in the situation.

4. Teach reciprocity

If people turn the other cheek and forgive each other 490 times or more, then this just allows people who exploit the suckers to prosper. If people repay kindness with kindness but also defection with defection, then this will help keep down the population of defectors and exploiters.

5. Improve recognition abilities

If you don't recognize the person who defected on you last time around, you won't be able to exact your revenge this time around. If you can tell ahead of time whether someone looks like a cooperator or a defector then you can be much better off rather than going in blind.


In Chapter 8, Axelrod looks at some ways in which a basic scenario of
randomly occurring interactions can be modified.
4 different types of 'structure' are considered.

1) Stereotyping - stereotyping means that you judge people based on some easily obesrvable characteristic such as the colour of their skin. Axelrod points out that these sorts of stereotypes can be self-reinforcing. If a blue person expects to be poorly treated by a green person (and vice-versa) then they have no reason to cooperate and will defect against each other. This defective behaviour then reinforces the notion that those blue/green people never cooperate. While this has negative consequences for everyone in that opportunities for cooperation are missed, it also has particularly negative consequences for whichever stereotyped group is in the minority since they will face a lack of cooperation from a majority of the population.

2) Reputations - Axelrod points out that it is good to have a reputation as a bully (someone who will respond to any defection with a very heavy retaliation) since that will scare people into cooperating with you. The hazards of establishing this sort of reputation is that you must pass up opportunities to engage in cooperative behaviour with people by forgiving them for their past transgression. And if more than one person is trying to establish a reputation as a bully, this can lead to a long series of defections until one gets the upper hand.

3) Government - The government needs to design the payoff structure such that most of the citizens will comply on their own. Punishment is generally reserved for setting an example of the few people who do defect and reassuring people that other people are getting away with breaking the laws. Once enough people began to ignore a particular law, because they don't respect the legitimacy of the law and the punishment/reward payoffs aren't set correctly, then enforcing compliance becomes extremely difficult because the cost is simply too high to physically coerce a large percentage of the population and the law tends to break down (see Prohibition, War on Drugs, Marijuana).

4) Territory - Axelrom segues from talking about government to talking about territory by noting that, 'an interesting characteristic of governments that has not yet been taken into account is that they are based upon specific territories.'

Axelrod develops a formal territorial model in which each participant in the situation has four neighbours, one to the North, South, East and West. Each round, a participant plays a repeated Prisoner's Dilemma against the four neighbours and is assigned a score based on their combined result against their four neighbours. Then for the next round, if a participant finds that one of their neighbours did better than they did, they switch to using their neighbours strategy. In this way, successful strategies can spread throughout the population.

Axelrod finds that in the territorial model, it is at least as hard (or harder) for a new strategy to invade a population using a given strategy (as compared to a mode where people meet randomly). If a strategy is stable (resistant to invasion by other strategies) in a population that mixes randomly, it will be stable as well in a population that is organized territorially.

Another finding was that in a territorial model, where people imitate their best neighbour, strategies that do really well in some situations and poorly in others tend to do better than they would in a model where people mix randomly. The territorial nature of the model means that you end up with a greater diversity of strategies in certain areas, and this allows the inconsistent strategy to thrive and convert its neighbours in the areas where conditions are suitable, while dying away rapidly where conditions are unsuitable.


Chapter 9 is a conclusion which basically just recaps everything that has
come before.

The quick summary of Axelrod's results is that, cooperation based on reciprocation (e.g. tit for tat style behaviour) can get started and can thrive in a wide variety of environments and that it withstand attempted 'invasions' by uncooperative strategies. Furthermore, the people doing the cooperating don't have to be friends, they don't have to possess foresight and they don't have to be rational. These characteristics might help, but they are not necessary as shown by the evolution of cooperation between enemy soldiers on the front in WWI, cooperation between bacteria and so on. Furthermore, neither altruistic behaviour nor a central authority is required to maintain cooperation (at least in Axelrod's model where people still retain the capacity to retaliate after someone has defected against them).
Axelrod notes that Prisoner's Dilemma situations ...

As always, there is far more to the book, then I covered here. The Evolution of Cooperation is noteworthy for the clear prose of the author and the thorough take on one particular type of interaction, the repeated Prisoner's Dilemma with two participants. I recommend anyone interested enough in the topic to have made it to the end of this post, should read it for themselves (if they haven't already).

61. The Evolution of Cooperation (part 1 of 2)

Note: This post is the sixty-first in a series about government and commercial ethics. Click here for the full listing of the series. The first post in the series has more detail on the book 'Systems of Survival' by Jane Jacobs which inspired this series.

The Evolution of Cooperation is the title of perhaps the most famous book on the Prisoner's Dilemma, and possibly Game Theory in general, ever written - by Robert Axelrod.

Reading it again, for the first time in a long time, I could see why it is so popular - it manages to cover a lot of ground with very clear, accessible prose.

The Evolution of Cooperation starts off by recounting a famous game theory tournament. Participants were invited to submit a strategy or 'rule' that would play a Prisoner's Dilemma against strategies submitted by other people. The strategies would be paired up against each other in turn and would play a repeated Prisoner's Dilemma against each other for a certain number of times. The goal was to achieve the highest possible point total, adding up across the matches against all the other strategies.

Recall that the nature of the Prisoner's Dilemma is such that, no matter what action your opponent takes, you will maximize your own total by defecting rather than cooperating. But by changing the situation from a single game to a repeated game, and by allowing participants to retain a memory of what happened before and by allowing them to clearly identify who they were playing against, the tournament introduced a strong signalling element into the Dilemma.

The tournament was won by the simplest strategy submitted, a strategy known as 'Tit For Tat.' Tit for Tat started off by cooperating (Axelrod refers to strategies that start by cooperating as 'nice' strategies), and then each round it just reacts to what the strategy it is matched up with did the previous round. If the strategy it is playing with defected on the last round, Tit for Tat defects this round, and if the strategy it is playing against cooperated on the last round, Tit for Tat cooperates this round.

After the results of the first tournament were published, a second one with more entries was held, but Tit for Tat again turned out to be the winner.

Strategies aren't fixed over time, and people might change their approach if they see another approach that is working better. Or those using a poor strategy might die out (or get fired) and be replaced by someone with a better strategy. Or some people may simply decide to try a new approach that they thought up. Through these sorts of mechanisms, the distribution of strategies, or rules, being used in the population can evolve over time.

An evolutionarily stable strategy is one that, even if everybody in a population is using it, can't be invaded by some other strategy designed to take advantage of it. Axelrod notes that a population where everybody defects is evolutionary stable because it is not possible for anyone playing any sort of cooperative strategy to invade (because they never meet anyone who will reciprocate their cooperation). But even a small cluster of cooperators can invade a much larger population of defectors if the conditions are right (because they will do well enough cooperating with each other to offset their poor results against the defectors).

But the converse is not true. A population where everybody plays a nice strategy like Tit for Tat can't be invaded by an 'Always Defect' strategy, because the Tit for Tats will do better playing each other than the 'Always Defect's will do playing with each other. This is a hopeful result (for those who like to see cooperation) since it suggests that a cooperative equilibrium is more stable than a defective one and that even a small group of cooperators can sometimes thrive in a sea of defectors.

Based on the results of the tournaments, and the success of Tit for Tat, Axelrod offers the following suggested courses of action for doing well in a repeated Prisoner's Dilemma type situation:

1) Don't be envious

As we saw before, envy can transform an absolute gain into a relative loss and a positive sum situation into a zero-sum situation. A common theme throughout the book is the distinction between absolute gains, made possible by the non zero-sum nature of the Prisoner's Dilemma, and zero-sum situations where only relative gains are possible.

2) Don't be the first to defect

'Nice' rules which don't defect first, will do well when playing with each other. This means that 'Mean' rules which defect first, will end up with lower scores against 'nice' opponents than 'Nice' rules do.

3) Reciprocate both cooperation and defection

A failure to reciprocate cooperation leads to unnecessary defection on both sides. A failure to reciprocate defection (by defecting in return the next round) leads to being taken advantage of.

4)Don't Be Too Clever

Unlike in a zero-sum game where you don't want your opponent to have any advantage, in a Prisoner's Dilemma it is important that those who are willing to cooperate recognize that you are willing to cooperate as well. Tit for Tat is a simple rule that helps other rules understand what they are dealing with and act accordingly. And since the best plan when facing Tit for Tat is to cooperate, rules will generally cooperate when they figure out that is the rule their opponent is using.

* * *

Moving along, Chapter 4 shows that friendship is not necessary for cooperation to develop by recounting the story of the 'live and let live' system that developed in the trenches during World War I where enemy units would cooperate by not killing each other, while facing off with each other across the same piece of ground for months at a time.

Chapter 5 shows that even creatures with very limited intelligence (e.g. bacteria) can engage in cooperation in Prisoner's Dilemma type situations. It also theorizes that the cooperation born from Kin Selection (the notion that it makes sense for us to evolve so that we are willing to make sacrifices for those we share genes with) might have provided a foothold of cooperation that could have spread into the sort of reciprocal tit for tat cooperation that would extend across larger groups of people, regardless of whether they are related or not.

I'll cover the rest of 'The Evolution of Cooperation' and talk about some of the implications of the ideas covered in it in next week's post.