Adam Ozimek speculates that Big Data might bring about more price discrimination. First degree price discrimination has always been a sort of business holy grail, but it was too difficult to get enough information on the shape of the price-demand curve to make it so.

For some time now, though, this has no longer been the case for many companies, and in fact one company did try to capitalize on this: Amazon.com. I know because I was there, and the reason that was a short-lived experiment is a real world case study of how the internet both enables and then kneecaps this type of price discrimination.

Amazon, until then, had one price for all customers on books, CDs, DVDs (this was the age before those products had been digitized for retail sale). A test was undertaken to vary the discount on hot DVDs for each customer visiting the website. By varying the discount from 10% up to, say, 40%, then tracking purchase volume, you could theoretically draw the price-demand curve with beautiful empirical accuracy.

Just one catch: some customers noticed. At that time, DVDs were immensely popular, selling like hotcakes, and the most dedicated of DVD shoppers perused all the online retail sites religiously for the best deals, posting links to hot deals on forums. One customer posted a great deal on a hot DVD on such a forum, and immediately some other respondents replied saying they weren't seeing the discount.

The internet giveth, the internet taketh away. The resulting PR firestorm resulted in the experiment being cancelled right away. Theoretically, the additional margin you could make over such price discrimination is attractive. But the idea that different customers would be charged different prices would cause such distrust in Amazon's low price promise that any such margin gains would more than offset by the volume of customers hesitating to hit the buy button.

Ozimek notes this: "The headwind leaning against this trend is fairness norms." What's key to this is that the internet is the world's most efficient transmitter of information, and while it enables a greater degree of measurement that might enable first degree price discrimination, it also enables consumers to more easily share prices with each other. This greater transparency rewards the single low price strategy.

It's not a coincidence, in my mind, that Apple fought for a standard $0.99 per track pricing scheme with the music labels while Amazon fought the publishers for a standard $9.99 pricing for Kindle ebooks. Neither Amazon or Apple was trying to profit on the actual ebooks or digital music retail sales (in fact many were likely sold at break-even or a loss), they were building businesses off of the sale of complementary goods. In the case of Amazon, which is always thinking of the very long game, there are plenty of products it does make a healthy profit off of when customers come to its site, and getting users to invest heavily into building a Kindle library acted as a mild form of system lock-in. In the case of Apple, it was profiting off of iPod sales.

In the meantime, second and third order price discrimination continues to exist and thrive even with the advent of the internet so it's not as if the pricing playbook has dried up.

A skeptic might counter: didn't Ron Johnson get fired from J. C. Penney for switching them over to an everyday low price model? Didn't their customers revolt against the switch from sales and coupons and deals you had to hunt down?

Yes, but everyday low pricing isn't a one-size-fits-all pricing panacea (as I wrote about in reference to the Johnson pricing debate at J.C. Penney). For one thing, there is path dependence. Once you go with a regular discount/deal scheme, customers create a mental price anchor that centers on that discount percentage and absolute price. It's hard to lift an anchor.

J. C. Penney was trying to go from a heavy sale-driven pricing scheme to an everyday low pricing model, and that's an uphill, unmarked path. Only the reverse path is paved. It's not clear whether the switch would have worked in the long run. Johnson ran out of runway from his Board soon after he made the switch and revenues declined.

Everyday low pricing tends to work best when you're selling commodities since those items are ones your customers can purchase many places online. At Amazon we were far more interested in dominating one crucial bit of mental math: what website do I load up first if I want to buy something? We were obsessed with being the site of first resort in a consumer's mind, it was the core reason we were obsessed with being the world's most customer-centric company. Anything that might stand in the way of someone making a purchase, whether it be prices, return policy, shipping fees, speed of delivery, was an obstacle we assaulted with a relentless focus. On each of those dimensions, I don't think you'll find a company that is as customer-friendly as Amazon.com.

Ultimately, customers have a hard time figuring out intrinsic value of products, they're constantly using cues to establish a sense of what fair value is. Companies can choose to play the pricing game any number of ways, but I highly doubt Netflix and Amazon will choose to make their stand on the first order price discrimination game. There are many other ways they can win that are more suited to their brand and temperament.

Still, the peanut gallery loves to speculate that Amazon's long term plan is to take out all of its competitors and then to start jacking up prices. A flurry of speculation that the price hikes had begun spun up in July this year after an article in the NYTimes: As Competition Wanes, Amazon Cuts Back Discounts. After the NYTimes article hit, many jumped on the bandwagon with articles with titles like Monopoly Achieved: An invincible Amazon begins raising prices.

If you read the NYTimes article, however, the author admits "It is difficult to comprehensively track the movement of prices on Amazon, so the evidence is anecdotal and fragmentary." But the article proceeds onward anyhow using exactly such anecdotal and fragmentary evidence to support its much more certain headline.

Even back when I was at Amazon years ago we had some longer tail items discounted less heavily than bestsellers. However, pricing the long tail of books efficiently is not as easy as it sounds, there are millions of book titles, and most of the bandwidth the team had for managing prices was spent on frontlist titles where there was the most competitive pressure. All the titles listed in the NYTimes article sound to me like examples of long tail titles that were discounted too aggressively for a long period due to limited pricing management bandwidth and are finally being priced based on the real market price of such books. Where in the real world can you find scholarly titles at much of a discount?

The irony is that the authors cited in the article complain their titles aren't discounted enough, while publishers ended up in court with Amazon over Amazon discounting Kindle titles too much. This is to say nothing of the bizarre nature of book pricing in general, in which books seem to be assigned retail prices all over the map, with the most tenuous ties to any intuitive intrinsic value. The publishers set the retail price, then Amazon sets a price off of the retail price. If the publishers wants the discount on their books to be greater they could just increase the retail price and voila! The discount would be larger.

To take another category of products, DVDs, soon after we first launched the DVD store, long tail title like Criterion Collection DVDs were reduced from a 30% discount to a 10% to 15% discount. But just now, I checked Amazon, and most of its Criterion DVDs are discounted 25% or more. If I'd taken just that sample set I could easily write an article saying Amazon had generously decided to discount more heavily as part of its continued drive to return value from its supply chain to customers.

Could the net prices on Amazon be increasing across the board? I suppose it's possible, but I highly doubt that Amazon would pursue such a strategy, and any article that wanted to convince me that Amazon was seeking to boost its gross margins through systematic price hikes would need to cite more than just a few anecdotes from authors of really long tail books.

It will remain a tempting narrative, however, because most observers think it's the only way for Amazon to turn a profit in the long run.

However, that's not to say big data hasn't benefitted them both in extraordinary ways. Companies like Amazon and Netflix know far more about each of its customers than any traditional retailer, especially offline ones, because their customers transact with them on an authenticated basis, with credit cards. Based on their customers' purchase and viewing habits, both companies recommend, better than their competitors, products their customers will want.

Offline retailers now all want the same type of data on their customers, so everyone from your local drugstore or grocery store to clothing retailers and furniture stores try to get you to sign up for an account of some sort, often by offering discounts if you carry a free membership card of some sort.