The origins of "The Long Tail"

One of the signs of a great idea is that people feel like they've known it forever. That, at least, is what I tell myself when people suggest that the concept of The Long Tail predates my article of the same name last year. This first came up in the crafting of the Wikipedia entry on The Long Tail following my original article, and it again rose in my conversations with Tom Standage of The Economist last week. I'll take a moment here to explain the derivation and perhaps we can clear up this issue once and for all.

It is not new that powerlaw (and other) distributions have "heads" and "tails". It is also not new that some tails are longer than others. Indeed, if you search for "long tail" in the context of statistics (as opposed to animals with long tails and long tail boats [shown])  before September 2004, when my original piece came out, you will find quite a few references to various curves that have long tails. But the only one that even comes close to talking about the consequences of the abundance boom created by technology is a passing reference in a 2003 essay by Clay Shirky about powerlaws distributions in the blogosphere ("Meanwhile, the long tail of weblogs with few readers will become conversational...").

Given my intellectual debt to Clay for his groundbreaking powerlaw research, I was delighted to see him comment on this in the discussion behind the Wikipedia entry. He writes:

"For my part, when I used the term in "Powerlaws, Weblogs, and Inequality", I didn't think of it as a coinage at all -- linear distributions have heads and tails and the weblog tail is long and flat....Chris and I and lots of other people use the phrase to describe a particular kind of distribution, but Chris has taken it in the direction of Tipping Point, a phrase that conjures up a whole complex of related issues, particularly issues of the business aspects of media and culture, that I didn't. So from my pov, Chris should get credit for originality, not of the phrase but of its current application and vividness.

To be precise, what I coined was the notion of looking at the tail itself as a new market. The use of the proper noun (including "The") is not incidental, but is intrinsic to the observation that we have historically looked at the market at the head of the curve in isolation, and we can now shift our gaze to the right and see that the tail is another market.

The notion of two markets--The Head and The Long Tail; one familiar, the other long ignored but now emerging--is at the core of the thesis and explains the initial-caps TLT construction I've used. To get a sense of its resonance, compare  a long tail search before September 15th, 2004 (1,570 results) with one after (619,000 results). Long tails are not new; The Long Tail is.

Comments

[Karmakin]

It was on the tip of everybodies tounge, we just gave it a name...

In my circles we talked about something similar, called it the Napster effect. It had two parts. #1. Basically what is the Long Tail effect, people can find and enjoy new music that by the old rules they never even would have known existed, and #2. A tail within that band itself, where your downloads wouldn't be limited to album stuff (which you'd buy), but you'd spend time downloading rare b-sides, live concerts, alternate takes, and so on.

The market always existed, more or less, check out any decent used CD store to find that out, but digital marketing through P2P really exploded it.

[Kevin Marks]

Chris, by capitalising this and particularising it you did help give it wider currency, but I think there were quite a few of us in the Long Tail saying this before you moved it into the Big Head.

I pointed out the zipf nature of music in 2002, compared distributions in different markets in Feb 2003 and spoke of the economic long tail at Bloggercon 2003.
Not to mention Gary Wolf's Wired article on Amazon in October 2003 which said:

All of Amazon's important innovations - starting from the concept of a Web bookstore - have suggested a profound change in the bookselling business, a change that makes it possible to earn a profit by selling a much wider variety of books than any previous retailer, including many titles from the so-called long tail of the popularity curve. 'If I have 100,000 books that sell one copy every other year,' says Steve Kessel, an Amazon VP, 'then in 10 years I've sold more of these, together, than I have of the latest Harry Potter.'

All credit to you for the brilliant "Long Tail" formulation, which moved us away from the Gramscian connotations of "Power Law" that bedevilled previous discussions. Naming things well does have power.

[Rajan Lukose]

I liked the way you used the number of search results to make the point about the impact of your article (which I have found to be enormously influential...amazingly so). It would not have diminished your point to have stayed consistent by using AllTheWeb.com, for results after Sept. 15, 2004, even though it would have given you "only" 57,600 results (instead of the 619,000 from Google).

Actually, it looks like you may have forgotten to include the term "statistics" in your Google search (which was used in the AllTheWeb search). This would have given 63,700 results on Google in the past year (pretty close to the AllTheWeb estimates).

[Cardozo Bozo]

You call it "the Head"? I've been calling it "the Tall Spike." Sure, powerlaws have been around a while and attracted a vocabulary, but "the head" if pretty generic. There are lots of "heads." "Long Tail" is unique; stands out in a conversation. Calling something a Tall Spike story, search term, novel, movie, etc. - that means something.

[Cardozo Bozo]

Heh. Just had a thought.

Pre-Industrial Revolution - the limitations were knowledge and manual power. There were plenty of advantages to making everything the same (don't have to think as hard), but not enough advantages to make it a market winner. People like personalization, so if you go to a cobbler he can put some tassles on it, because the personalization doesn't add much to the marginal cost.

This is economy is slightly more 'long tail' than 'tall spike.'

Industrial Revolution - marginal cost of manual strength falls to almost zero. Just add more steam/ coal/ oil. Machines can make things, but they can't personalize. The "sweet spot" of busines is making the cheapest pattern that will sell to the most people. It started in commodities (Rockefeller) because commodities don't need a lot of personalization, but it moved into appliances later (GE, Ford). The Model-T is the "Tallest Spike" of automobiles, but he went too far up the Spike. General Motors showed that a little more personalization was good, and had lots of models. Ford promptly climbed down the Spike a bit, until the industry reached equilibrium and hit diminishing returns on demand for personalization in automobiles.

Post-Industrial Age - Cars are still coming down that Spike. Toyota's Scion is the lowest I know of, with modularity designed in and the after-market customization crowd actively anticipated.

I've got some more to say, but I don't know how to say it. Can't quite get it out.

[Chris A nderson]

Rajan,

I was waiting for someone to bust me on that. I think there's a glitch in the date function of alltheweb that excludes search results with ambiguous dates (such as those dynamically generated). Virtually all of them are post the article, as you can see if you compare the two search result sets, but only Google is relaxed enough in its date criterea to include them.

I actually linked to a "past year" seach on Google because they don't offer a "past nine months" search, which would have corresponded to the actual publication date better, but I don't think it's far off. Good enough for web work.....

[David]

Heads and Long Tails seem to go together, and it at least keeps the metaphor consistent. The Tall Spike may describe part of the distribution curve well, but I think it would confuse the discussion and be somewhat less memorable in this context.

[hugh macleod]

Well, I'm a huge Clay Shirky fan, and I know well both the article and the phrase ("the long tail") you speak of.

That being said, I give you all credit taking three little words, or at least, a small thought, and turning it into a very big idea.

[Robert Steers]

Very good article! I had heard this term used before by Chris Patten (http://europa.eu.int/comm/mediatheque/photo/commprodi/albpatten_en.htm) the last british governor of Hong Kong, in about 2000. He was using the term to describe the fact that England has a "long tail" of people with below average education.

Certainly the long tail presents some interesting marketing challenges, but the point of the long tail is it has always been there. An example would be some of the book stores near Leicester Square t(london) that sell nothing but art books printed before C19th. They have been there for donkey's years.

I suppose with the internet, within this long tail there is
1) increase competition for niche markets, as everyone can now have a stab at it,
2) increased choice for the consumers in those small markets[would you like to go to the store that specialises in yellow wooden penny farthings or the red kind],
3) what is percieved to be more trash by everyone outside those markets.

What do you guys think?

[David]

Robert, I'd add to that:
4) increased access to things that many people want, but that the big producers/distributors haven't figured out that there's a market for.

[Kevin Marks]

I thought about this some more, and blogged about how to profit from the really long tails

[murky]

Your staking out of a claim for coining a Webby word is only the non-monetary side of the Web economy. The other side is prestige--which is what people work for in academia. Even when the dot-com bubble burst, the Web continued to suck into it "venture creativity," which is because it created a vast new market for prestige just waiting to be tapped. While there's only so many places for people on reality TV, there a limitless number of spots in the blogosphere. You pointed that out with regard to making money, but lots of people are here just to make their names. Money's good for food and shelter but prestige is the capital that most naturally converts to sex and power. If somebody invents a currency for prestige we'll really be off and running. Just remember you heard it here first.

[murky]

"Your staking out of a claim for coining a Webby word is only the non-monetary side of the Web economy."

Er..whoops: Make that "Your staking out of a claim for coining a Webby word ILLUSTRATES THE the non-monetary side of the Web economy."

[John Arenivar]

Of course there are other non-web references to long tails in statistics, the most famous being Stephen Jay Gould's essay "The Median Isn't the Message" (reproduced here: www.cancerguide.org/median_not_msg.html)
In an interview with Salon in Aug. 1996 (www.salon.com/weekly/interview2960923.html) I find this quote:" Q: When I finished "Full House" I started to think of other directions you could turn this way of thinking toward: the book market, for example. We tend to focus on best sellers or the books we think are best written, but if you study the spread of variation in its population, you'd probably pay a lot more attention to romances and self-help books.

Gould: So often, we characterize a whole system by an extreme value that intrigues us. And that value often is moving somewhere, simply because of the variation -- an extreme value's gotta move if the variation's expanding and contracting."

SJG called his view of statistics a "right tail", but it's basically the same idea. His applications were to a) cancer statistics and b) evolutionary theory, not economics.
I don't fault you for trying to identify yourself with an important idea, but you should give credit to those who came before.

[Frank Hirsch]

Arrggghhh!
Are some people just not able to realise that any idea that gets popular must have been around in various proto-stages for some time. Just as we can't learn what we don't already almost know. The decisive step may be minor, but that won't make it less decisive. Good promotion also helps, of course.