Powerlaw Primer

I'm re-reading Clay Shirky's excellent early work on powerlaws and am reminded that many people are still confused about what exactly a powerlaw is and under what circumstances it arises.

The first answer is easy: it's basically any curve that has a y=1/x sort of shape, like the chart on the right. You can find powerlaws practically anywhere you look, from biology to book sales. The Long Tail is a powerlaw that isn't cruelly cut off by bottlenecks in distribution such as limited shelf space and available channels.

The second answer is easy, too. Powerlaws come about when you have three conditions: 

1. Variety 
2. Inequality 
3. Network effects (word of mouth, for example) to amplify the differences between them.   

In others words, powerlaw distributions occur where things are different, some are better than others, and network effects can work to promote the good and suppress the bad. This results in what Vilfredo Pareto called the predictable imbalance of markets, culture and society: success breeds success, rich get richer and so on. Needless to say, these forces describe a good fraction of the world around us.

Comments

[Randall Newton]

Chris, I was fascinated by your original Wired article, and have been following LT since. I remember in the dotcom boom days, everybody talked about S-curves for product implementations. Is there a fundamental relationship between the S-curve and the LT curve?

[chris anderson]

Randall,

As as best I understand the S curve, it's unrelated. It's primarily a learning curve that takes a while to get traction, climbs steeply, and then levels off. No connection to the above, unless I'm really missing something.

[Kevin Marks]

Chris, power law distributions occur lots of other places too, in nature as well as in human-mediated areas. I can recommend "Ubiquity : Why Catastrophes Happen" by Mark Buchanan) for a discussion of this, including earthquakes and cities as 2 other examples.
Mandelbrot's Fractal Geometry of Nature discusses scale-free distributions at some length too.

[John Thacker]

The rules you've stated, however, are not sufficient for a power-law distribution. They are hypothesized as explanations for why we appear to get heavy-tailed distributions when studying Internet effects, but have not been conclusively proven. There are many, many distributions which look similar to power-laws, at least with small amounts of data. Power laws look like best fits in many cases, but there are other possibilities. One can look at the immense amount of discussion I saw at a recent conference at Cornell on heavy tailed distributions, and disagreement over whether particular phenonema actually had them.

An exponential distribution can look like a Pareto; the difference occurs out in the tails, and thus it takes a lot of careful data in order to be sure that one has a Pareto and not an exponential or something else. There are lots of non-heavy tail distributions.

Your last paragraph is confusing. Even in the non-heavy tail distributions there are network effects, success breeds success, and so on. It's possible that network effects can happen in such a way that the area under the tail becomes so small that it's not significant. That the Internet and everything else will not cause a revolution because the tail is too small. That things are like
ketchup, not mustard.

[T Neville]

Chris

I think Clay’s passing reference to the psychic allure of “solidarity goods” – of reading the same blogs as your friends – reveals a lot about why audiences cluster around a small number of available information/entertainment products.

When talking about media, especially electronic media, the “currency” value of sharing the same media experience at the same time has substantial social and commercial cachet.

This is why broadcast media had such a profound impact on social organization. It’s also why advertisers eagerly pay more (on a CPM basis) for large broadcast audiences who see their ads at the same time.

Marshall McLuhan recognized this too. He defined the term “mass media” as refering not to “the size of their audiences, but of the fact that everybody becomes involved with them at the same time.”

[Deus ex Machina]

I dont think its overly relavent if a distribution is strictly pareto or only exponential. the point is that vast inequality of outcome arise from incremental changes in differentiation.

the driving factor behind this outcome are really only 2 points: differentiation between producers and freedom of choice on the consumer side.

the efficiency of how information spreads seems to only have an effect on the time for a resulting pareto concentration.

[Manish Danani]

Hi,

I would like to know how the formula of powerlaw really works? What is the exponent that one uses and why? Would appreciate if you can throw some light on that.

Regards,

Manish
ERP & EAI Specialist
http://www.dcs.bbk.ac.uk/~ldana01/

[cheap eve isk]

I dont think its overly relavent if a distribution is strictly pareto or only exponential. the point is that vast inequality of outcome arise from incremental changes in differentiation.

the driving factor behind this outcome are really only 2 points: differentiation between producers and freedom of choice on the consumer side.

the efficiency of how information spreads seems to only have an effect on the time for a resulting pareto concentration.