Is the Long Tail full of crap?

One of the most frequent mistakes people make about the Long Tail is to assume that things that don't sell well are "not as good" as things that do sell well. Or, to put it another way, they assume that the Long Tail is full of crap. After all, if that album/book/film/whatever were excellent, it would be a hit, right?

Well, in a word, no. Niches operate by different economics than the mainstream. And the reason for that helps explain why so much about Long Tail content is counterintuitive, especially when we're used to scarcity thinking.

First, let's get one thing straight: the Long Tail is indeed full of crap. But it's also full of works of refined brilliance and depth--and an awful lot in between. Exactly the same can be said of the Web itself. Ten years ago, people complained that there was a lot of junk on the Internet and, sure enough, any casual surf quickly confirmed that. Then along came search engines to help pull some signal from the noise and finally Google, tapping the wisdom of the crowd itself to turn a mass of incoherence into the closest thing to an oracle the world has ever seen.

On a store shelf or in any other limited means of distribution, the ratio of good to bad matters because it's a zero sum game. Space for one eliminates space for the other. Prominence for one obscures the other. If there are ten crappy toys for each good one in the aisle, you'll think poorly of the toy store and be discouraged from browsing. Likewise it's no fun to flip through bin after bin of CDs if you haven't heard of any of them.

But where you have unlimited shelf space, it's an infinite sum game. The billions of crappy web pages about whatever are not a problem in the way that billions of crappy CDs on the Tower Records shelves would be. Inventory is "non-rivalrous" and the ratio of good to bad is simply a signal-to-noise problem, solvable with information tools.

Which is to say it's not much of a problem at all. You just need better filters, such as recommendations and good search engines. The fact that screens 10 and beyond of your Google search results are unhelpful doesn't matters because screens 1-3 are so useful. The noise is still out there, but Google allows you to effectively ignore it. Filters rule!

The following is this expressed graphically. As you go down the Long Tail the signal-to-noise ratio gets worse. Thus the only way you can maintain a consistently good enough signal to find what you want is if your filters get increasingly powerful.

This leads to the key to what's different about Long Tails. They're not pre-filtered by the requirements of distribution bottlenecks and all that entails (editors, studio execs, A&R guys and Wal-Mart purchasing managers). As a result their components vary wildly in quality, just like everything else in the world.

One way to describe this (using the same language of information theory that brought us signal-to-noise ratios) would be to say that Long Tails have a "wide dynamic range" of quality: awful to great. By contrast, the average store shelf has a relatively narrow dynamic range of quality: mostly average to good (there's some really great stuff, but much of that is too expensive for the average retail shelf; niches exist at both ends of the quality spectrum).

So tails have a wide dynamic range and heads have a narrow dynamic range. Like this:

Note that you have high-quality goods in every part of the curve, from top to bottom. Yes, there are more low-quality goods in the tail and the average level of quality declines as you go down the curve. But with good filters averages don't matter. It's all about the diamonds, not the rough, and diamonds can be found anywhere.

Before I go on I should say a word about the confusing terms "high quality" and "low quality". They are, of course, entirely subjective. Here are some examples of criteria people might use to value content:

"High quality":

• Addresses my interests
• Well made
• Fresh
• Substantive
• Compelling

"Low quality":

• Not for me
• Badly made
• Stale
• Superficial
• Boring

Note that all of those are entirely in the eye of the beholder; there are no absolute measures of content quality. One person's "good" could easily be another's "bad"; indeed, it almost always is. I like economics papers, but I have friends who like Maxim. They find my stuff boring; I find their stuff superficial (except for those Jessica Alba pics). 

This is why niches are different. Your noise is my signal. If a producer intends something to be absolutely right for one audience it will by definition be wrong for another. The compromises necessary to make something appeal to everyone mean that it will almost certainly not appeal perfectly to anyone--that's why they call it the lowest common denominator.

Is South Park badly made or brilliant? Once upon a time some network exec had to answer that question for all of us before it could get distribution. Cable lowered the bar and now Netflix and the web are lowering it further. In a Long Tail world each of us answers the quality question for ourselves and the marketplace sorts it out.

All this leads to three counterintuitive lessons of the Long Tail.

1. Niche content can be of higher quality than hit content.
2. It doesn't matter how much junk there is around those gems; with good filters, the average level of quality is irrelevant.
3. You can charge more for high-quality niche content because it is so well-suited to its audience.  

Comments

[David]

Chris, it seems that something being a mainstream hit almost guarantees it will be rejected by certain niche markets. It definitely happens in music. The last time I was exposed to a top 40 radio station I found it to be totally unlistenable. Whatever the filter is that allows a song to climb the charts seems to almost guarantee I won't like it.

On the other hand, much of the music I do listen to is very popular among my friends, and among people w/ similar sensiblities. Is it possible to have "niche hits"? Are these higher up on the tail, or are they on their own curves?

[Greg Linden]

Great post, Chris. The long tail is full of crap, but one man's garbage is another man's gold. If you can match content to interested audiences, you can take advantage of the long tail.

Your signal-to-noise ratio graph points to another issue. If you increase the amount of crap without changing your filter, quality will tend to go down. When search engines boast about increasing index size, they inadvertently could be reducing overall search quality by exposing more crap to uninterested users.

[Bill Fischer]

In the realm of the traditional retail channels for filmed entertainment, the Long Tail is merely a signifier for catalog titles and is agnostic with regards to quality.

Diminished marketing support, limited shelf space, and crude merchandising capabilities all relegate quality older films (greater than a year) to the tail. From our consumer research and customer data, we have seen that simply making an older title available (providing shelf space) and offering basic filters is enough to break this paradigm by moving low quality newer releases down the tail and moving quality older titles towards the head.

Bill Fischer
www.dvdstation.com/blog

[LeegleechN]

(in response to previous)
I think that depends on your niche. If you belong to a relatively large group of people who like that music, then your content will be higher up the tail. A "niche hit" is just content particularly well suited to one niche. That means it will sell well within that niche but it may sell poorly with all the other people.

[LeegleechN]

To clarify, my other post is in response to the first post. A bunch of other people replied when I was composing my post.

[Jack Krupansky]

Are you using the terms "filter" and "recommendation" as synonyms? Your "Filters rule" link does. I agree that recommendations and trust networks are a great *social* form of filtering, but they're not the same as the technical forms of filtering that you get with Google. Google's page ranking is of course more oriented to short-tail popularity.

The problem with recommendations and trust networks is that they don't help you gain access to the very long tail stuff that your personal network has not reached yet.

Technical filters certainly have some significant potential to reach out to the very long tail stuff (including deep left field and out in the weeds), but still have a very long way to go.

For example, if I personally had written something that might be of interest to you, how would you ever know?

I would also note that one person's junk can be another person's treasure, especially when you're talking about long tail stuff for which many people don't yet have standards by which to judge "crap or not".

I once wrote about something I call GMWIMW - Give Me What I Might Want, a mythical filter which would help you continually expand your horizons in a way that doesn't require that you *know* in advance what you want.

-- Jack Krupansky

[David]

Jack, GMWIMW seems to exist to some extent at Amazon. I mean it's limited somewhat by the things I search for, buy, or put on my wish list, but it does suggest other related books I didn't know existed.

Another good source seems to be the bibliographies at the backs of the books I do read. If I like the author, I'll consider him/her a potential filter. This is very low-tech, of course, but it does make me aware of all kinds of books way down the tail.

[Martijn]

Ever thought about the fact that every spot on the tail is the start of a new tail? I just like to make it more difficult to understand;-) Think 3D...

[MCM]

The interesting thing for me is looking back through recent history to see how further-right producers have exploited (or been exploited) in an attempt to ripple their point left.

Sometimes the difference between niche and monster hit is just in the marketing and advertising venues, so (in theory) if you can take a cheesy sci-fi b-movie and promote it right, it could become the top of the tail. But then what? Would it proportionately increase the position of its fellow niche-ers up the tail?

Star Wars was only really meant to be a niche product, but ended up switching around the LT quite a lot. Were all the crap sci-fi movies in the 80s really rip-offs, or just George Lucas' colleagues, with a huge new visibility?

[ed nixon]

To be honest, I don't follow your conceptual leap to S/N rations and stronger filters; nor do I follow the 'quality variability' idea.

Whose idea of quality? Why is there a lower S/N ratio further out the long tail? Whose signal? Whose noise?

One of the points I've taken away from your writing so far is that roughly speaking everything is always available. If that is a reality of the long tail then the dimensionality of the X axis becomes what?

I think you may be overlaying too many conceptual notions on a relatively simplistic and powerful graphic. You may need to expand the graphical analogies or develop some sort of alternative coding scheme.

...edN

[Chris Anderson]

Ed,

Good questions. A few answers:

You're right that everyone registers a different s/n, but down the tail there are 1) more products, 2) more narrowly targeted products and 3) more amateur production, each of us will find that the ratio of products that suits us to products that don't rises. Your favored products are different than my favored products, but both us find more that don't suit us the further down the tail we go.

The x axis is rank, in order of sales/consumption. The y axis is the sales/consumption themselves.

[Barry Ritholtz]

Science fiction author TheodoreSturgeon once said "90% of science fiction is crap. That's because 90% of everything is crap."

Quality is an aberational rarity . . .

[Alex Rowland]

I agree with most everything in the post. What I disagree with, however, is the idea that you will be able to charge more for niche content. The environment in which long tail content really thrives is within an open distribution networks. Open distribution encourages publishers to participate (which gives you the breadth of content), but it also favors less regulations, less DRM, and open standards. In such environments it becomes very difficult to charge consumers cash for access to content. To see this in action, just look at the web in general and blogs, podcasts, internet video specifically. Free content thrives, paid content withers. This only gets worse as the network aggregates greater quantities of content. Eventually, the network is governed by slightly different rules. As Clay Shirkey put it so succinctly in his essay 'Fame vs. Fortune';

“If InstaPundit and Samizdata are both equally easy to get to, the relative traffic to the sites will always match audience preference. But were InstaPundit to become less easy to get to, Samizdata would become a more palatable substitute. Any barrier erodes the user's preferences, and raises their willingness to substitute one thing for another….Though each piece of written material is unique, the universe of possible choices for any given reader is so vast that uniqueness is not a rare quality. Thus any barrier to a particular piece of content (even, as the usability people will tell you, making it one click further away) will deflect at least some potential readers… Charging, of course, creates just such a barrier.”

Even niche content suffers this same fate as the network scales. Producers will instead receive compensation by converting audience time and attention into cash, not by charging the consumer directly. Advertising, sponsorships, and donations will be the currency of the long tail.

[Jon Garfunkel]

So, the main innovation of the long tail is that the content is no longer pre-filtered. It is still post-filtered, though.

[Jakob Nielsen]

For a lot of utility-oriented niche content, the demand curve is fairly inelastic, meaning that as price goes up, sales don't drop correspondingly. Basically, either you need it or you don't. And if you need it, the price of content is often irrelevant relative to the value of the problem it solves.

For example, if a report can sell for either $50 or $100, there will be less than twice as many sales at $50, so more money is made at $100.

Of course, there's a limit on the price for each type of content, which is why you can't price it at the actual value generated for the customer. For example, it's rare to find a book that costs more than $100 or a report that costs more than $5,000. Both of these two prices are trivial if the text gives you info that, say, increases the sales of an e-commerce site by 10% (assuming that the site sells, say, $1M/year before the improvement). And still, people wouldn't pay $1K for a book, even though they would pay that much for a report with the same text.

Anyway, empirically, niche content certaily can be sold at a higher price than broader content, and that's also necessary since (by definition) the number of sales is smaller, but it still requires significant resources to create the niche content. (In fact, in my business, it often requires *more* resources to generate niche content, because it involves more original research, whereas the broad content is stuff that's already known and just needs to be written down in an understandable manner.)

Back to the original topic of the post: It might be better to avoid the term "quality", because that's so highly context-dependent. Instead, maybe talk about "relevance" or "value" for the target audience. Or, at the minimum, specify that quality is judged by the target audience and not by others.

For example, an opera recording may be high or low quality according to the criteria of opera lovers. (Say, a world-class tenor vs. somebody who can hit the notes, but not spectacularly so.) Doesn't matter. In either case the recording will be detested by people who dislike opera.

[David]

Seems to me that the issue isn't about quality, relevance or value. It's really about the constraints of shelf space, and what happens when those constraints disappear.

[Alex Rowland]

Jakob. I think you're observations have historically been accurate. I think what is changing is how content is distributed to consumers. In a closed network environment, you are right, niche content pricing can be fairly inelastic (although not to the degree I think you argue). But this changes dramatically in an open distribution environment. In an open network everyone can contribute. There are none of the same limitations imposed by the physical, legal, or regulatory structures of the underlying network. In this environment, free content has an enormous advantage over paid content. And just being unique becomes less and less of a reason to fork over hard earned dollars. Most consumers would just rather get 90% (or more) of the value at zero cost.

In an open network, the demand curve is actually highly elastic and only becomes more so as the network matures. In an open network you're just as likely to find the content of those $5,000 reports for free.

[Paul Moore]

In regards to the content creator (artist, writer, filmmaker, what have you) I posted some thoughts on the implications of this little element of filters on my site
http://godinruins.typepad.com

[Marko]

Long Tail full of crap? Wth? I would say it's cool!

[Canada Without Prescription]

yes Long Tail is full of crap people and I want to emphasize this blog Excellent my friends have lots of views in the comments that I feel great, keep in that way.