Crawl Across the Ocean

Monday, February 13, 2006

The Olympics - Keeping Score

All else being equal, I prefer simplicity over complexity, but let's face it, all else is rarely equal. I have a bit of a theme on this blog of criticizing things which are flawed due to being oversimplified, with our electoral system - a simple but flawed way to translate voting intentions into representation, and GDP - a simple but flawed measurement of our economic status, being just a couple of examples.

Under the miscroscope today is the medal count, a simple but flawed way to compare the relative performance of nations at the Olympics.

I see three main problems with the medal count:

Problem 1) It ranks all medal types the same. For example, a country with 11 gold medals ranks behind one with 12 bronze medals, which doesn't seem right.

Now some people list the countries based first on number of gold medals but this is not satisfactory either. Under this system a country with just one gold medal ranks ahead of a country with 25 silver and 10 bronze (but no gold).

Solution: Countries should be awarded points for each medal received, with the points being distributed to reflect the fact that gold is better than silver and bronze, but not infinitely better. A 3-2-1 point system is the obvious choice, but I think this overstates the value of a gold vs. a bronze somewhat. Are 2 gold medals really equivalent to 6 bronze? Instead I prefer (purely subjectively) a 4-3-2 point system, under which 2 golds are equivalent to 4 bronze, not 6.

Problem 2) All events are weighted equally, but it would make more sense to rank all sports equally. The short explanation is to simply ask if biathlon should really count for 5 times as much as hockey in the medal standings.

A little bit more rigorously, it seems logical that if country A is good at Sport X and country B is good at sport Y then country's A and B are both good at one sport and roughly equal. Just because Sport X has 12 events (and hence 36 medals) and Sport Y only has 2 events (6 medals) doesn't make country A 6 times as good as country B.

Solution: Rather than adding points for each individual medal to the national total, the point totals from each sport should be normalized so that the total points available from each sport is the same. For example, a medal in Alpine Skiing (10 events) would only get 1/5 the weighting of a medal in curling (2 events).

Problem 3) Because the number of events and sports can vary from Olympics to Olympics, the medal counts are not really comparable over time.

Solution: Rather than computing a point total, national scores should be expressed as a percentage of the total points available.

Combining these three items, I will be using the following methodology to rank countries in the Olympics:

1. Assign 4 points for every gold medal, 3 for every silver and 2 for every bronze.

2. In each sport, compute the percentage of total points available for that sport that each country has achieved. For example, if the Americans were to get one gold, one silver and one bronze in Freestyle Skiing, then they would have 9 points out of a total 36 (36 because there are 4 Freestyle Skiing Events, and each event has 9 total points available) for 25% of the total freestyle skiing points.

3. To compute the total score for each country add up the individual scores in each sport for each country (so the U.S. score would be 0.25 (to represent the 25% in freestyle) + their percentage of the points in alpine skiing, + their percentage for biathlon, etc.) and then divide by the number of sports.

This methodology reflects the rank ordering of gold, silver, bronze, weights all sports equally and ensures that the total of all the national scores always adds up to 100%, even if the weightings for the different medal are changed or if the number of events changes of the number of sports changes. Each country's total will represent their percentage of the total points available.

When I get a chance, I'll post the standings for Turin, as per this methodology, perhaps with a comparison to the standard medal count.


  • While you're at it, let's see the revised methodology results for Salt Lake City!

    By Blogger Simon, at 5:14 PM  

  • Ideally, I'd like to do the revised results for every Winter Olympics and then do a chart of countries performance over time, but I doubt I'll ever find the time/motviation.

    By Blogger Declan, at 10:30 PM  

  • Declan, since you're normalizing like a fiend, you may as well normalize for population as well. A country like China has a much larger population base from which to find its sports superstars.

    By Blogger canukistan, at 10:23 AM  

  • Yes, that is another logical next step. But we have to be careful to keep separate the question of measuring how well a country performed with the question of how well they *should have* performed.

    There is an interesting paper on the topic of predicting medal results here It's a bit technical but the main upshot is that by far the strongest predictor of success is not population or per capita wealth, but rather the combination of the two (total GDP), reflecting both the larger pool of potential athletes and the greater resources available to each athlete.

    So I guess I could add on a final step of normalizing for national GDP, but then again, the Norwegians are already winning, we don't want thigns to get really ugly!

    By Blogger Declan, at 12:46 PM  

  • I've thought about this before, but there are lots of events that are ambiguous as to whether to combine. Luge and skeleton? Short track and long track speed skating? Alpine skiing and freestyle skiing? Possibly also snowboarding? Biathalon and cross country skiing? And worst of all, those last five and nordic combined?

    There are certainly arguments for and against combining some of those, and I don't know enough about the sports to decide which are correct.

    By Anonymous Anonymous, at 8:46 AM  

Post a Comment

<< Home