Tuesday, 23 August 2011

Popular inaccuracies: A review of Malcom Gladwell’s “The Tipping Point”

Malcom Gladwell’s book The Tipping Point has itself become something of a fad. It is hugely popular and this might lead one to believe that the author really has discovered the social laws that explains why some things catch on, while other books – or diseases, or rumours, or habits – seem to stumble before they get off the ground. Is this true?

Despite the popularity of the book I remain sceptical of much of its content. First of all, the arguments are far too anecdotal to be convincing. Anecdotes makes for easy and entertaining reading. However, precisely because it is so easy to remember anecdotes they can convince us when we should be sceptical. Second, one of the major messages is at best an oversimplification and at worst very misleading. It is simply not true that it is necessary to have some highly connected individuals to get an epidemic going. Third and finally, at some points – such as the importance of zero-tolerance in fighting crime - detailed investigations reveals more doubts than the enthusiastic picture that is painted in the book.


A brief summary

How do epidemics – social and real – get going? The answer, Gladwell claims, is three principles. First, he assigns a crucial role to some individuals. The most important individual would be someone who are well-connected (have many friends, sexual partners and so on), are knowledgeable, and those who are able to sell the message. Second, the thing that is to be spread has to have “stickiness.” If it is information, it has to be easily remembered. Finally, the context can sometimes be such that an epidemic spreads easily, for instance by encouraging certain kinds of behaviors.

Obviously there is some truth to these arguments. Gladwell’s main point, however, is simply not just that these are important principles- He also argues that the theory unite a lot of rather diverse phenomena and that it shows how small things can have large consequences. The spread of rumours, HIV, crime, syphilis, smoking and lots of others phenomena are used to illustrate the importance of the three principles and in all cases he argues that small changes can have large effects. For instance, small things making a message glue itself easier to the brain can make a a large difference to the sale of a product.

How are we convinced?

To convince the reader of his message, Gladwell gives us illustrative examples. He tells us about the homosexual flight-attendant who supposedly had 2500 partners, about a successful businessman who had an extremely large circle of friends, and how the spread syphilis in a city could be traced back to a small group of individuals. To be fair, Gladwell also cites scientific research. The reader is told about a wide range of experiments and results. However, the research is used selectively. He uses articles that support his views without pausing to mention potential weak spots or rival views. In sum, the combination of anecdotes and scientific references can be quite convincing at first, but on closer inspection they tend to reveal several weaknesses. A good example includes the role he assigns to “special” individuals.


The role of special individuals

The flight attendant with many sexual partners is used to convince us that some extremely well-connected individuals are crucial to the spread of diseases. It is clearly true that such individuals could spread a disease rapidly. Still, we must ask whether the existence of these individuals really are necessary and important for the quick spread of a disease. It turns out that they are not. To understand this, think first about a community in which everybody knows their neighbours and nobody else. In this imagined community everybody has the same number of friends and there are no special individuals in the sense of a person who is extremely well connected. In such a community it would take some time for a disease to spread from one random person to another because it would have to go through the whole chain of neighbours.

Now imagine that we create some friendships across the community so that our inhabitants meet some people in addition to the neighbours. There might be some differences in the number of friends each individual has, but this is not important. The important point is simply that we allow some more or less random connections between people other than to their neighbours. It turns out that in such a community a disease or a rumour can spread very quickly (the small-world property). Note here that the key is not that there are some people who are extremely well connected. It is the fact that there are some cross-cutting connections that is important for the speed by which something spreads. Reading the Gladwell’s book one gets the opposite feeling: That some individuals are extremely important. This is misleading in an important way. It makes us give more prominence to individuals than we should and less to the structure of the system.

The devil is in the details

The spread of crime can be viewed as an epidemic and the question then becomes what it is that is driving the epidemic. In this book Gladwell seems to favour the so called zero-tolerance or “broken-glass” argument and he cites researchers who have argued for this. However, while it is intuitive that our surroundings affect us and out behaviour, closer investigation reveals that there are weaknesses in the empirical evidence connecting the drop in crime with the introduction of zero-tolerance policies. In several places the drop appears to have started before the policy was investigated. Also, while Gladwell may be excused for not knowing about Lewitt’s work on the potential importance of abortion in explaining this drop, he must have known that there are many alternative possibilities that we should explore before simply accepting the zero-tolerance argument just because it fits the argument so well (that small things can have big effects).

In sum

One might complain that it is unfair to use such strict criteria for a popular science book. I sympathize with this. One clearly cannot write a book with lots of statistics and expect to sell a lot of books in airports. One could also argue that at least the book stimulates the mind and may serve as an appetizer to more rigorous arguments. Still, there are lots of popular science books that manage to be critical and popular without compromising too much on substance. Simon Singh’s book Fermat’s Last Theorem is an excellent example of this. Although not a bad book, I think The Tipping Point ends up compromising too much on substance.

Thursday, 4 August 2011

Convert text to numbers in excel

Excel is nice (much better than word), but sometimes it drives me mad. Like today. I found a table with numbers on health expenses from 1946 until 2006 on a web page and wanted to use the numbers in some calculations. Copied the whole thing into excel. Everything works fine, except excel believes the numbers represent a text string. This is a common problem because the numbers had an initial space and a space as a thousand-delimiter. Now, there are some solutions (trim, clean and so on), but they did not work in this case. And just to make matters worse it is not possible in excel to search for a blank spaces and replace them with nothing. Annoying! Solution? Well copy the table to word, search and replace all spaces with nothing, copy the table back to excel. Voila!