Moravec's Paradox and self-driving cars

Moravec's Paradox:

...the discovery by artificial intelligence and robotics researchers that, contrary to traditional assumptions, high-level reasoning requires very little computation, but low-level sensorimotor skills require enormous computational resources. The principle was articulated by Hans MoravecRodney BrooksMarvin Minsky and others in the 1980s. As Moravec writes, "it is comparatively easy to make computers exhibit adult level performance on intelligence tests or playing checkers, and difficult or impossible to give them the skills of a one-year-old when it comes to perception and mobility."[1]

Linguist and cognitive scientist Steven Pinker considers this the most significant discovery uncovered by AI researchers. In his book The Language Instinct, he writes:

The main lesson of thirty-five years of AI research is that the hard problems are easy and the easy problems are hard. The mental abilities of a four-year-old that we take for granted – recognizing a face, lifting a pencil, walking across a room, answering a question – in fact solve some of the hardest engineering problems ever conceived... As the new generation of intelligent devices appears, it will be the stock analysts and petrochemical engineers and parole board members who are in danger of being replaced by machines. The gardeners, receptionists, and cooks are secure in their jobs for decades to come.

I thought of Moravec's Paradox when reading two recent articles about Google's self-driving cars, both by Lee Gomes.

The first:

Among other unsolved problems, Google has yet to drive in snow, and Urmson says safety concerns preclude testing during heavy rains. Nor has it tackled big, open parking lots or multilevel garages. The car’s video cameras detect the color of a traffic light; Urmson said his team is still working to prevent them from being blinded when the sun is directly behind a light. Despite progress handling road crews, “I could construct a construction zone that could befuddle the car,” Urmson says.

Pedestrians are detected simply as moving, column-shaped blurs of pixels—meaning, Urmson agrees, that the car wouldn’t be able to spot a police officer at the side of the road frantically waving for traffic to stop.

The car’s sensors can’t tell if a road obstacle is a rock or a crumpled piece of paper, so the car will try to drive around either. Urmson also says the car can’t detect potholes or spot an uncovered manhole if it isn’t coned off.

More within on some of the engineering challenges still unsolved.

In his second piece, at Slate, Gomes notes something about self-driving cars that people often misunderstand about how they work (emphasis mine):

...the Google car was able to do so much more than its predecessors in large part because the company had the resources to do something no other robotic car research project ever could: develop an ingenious but extremely expensive mapping system. These maps contain the exact three-dimensional location of streetlights, stop signs, crosswalks, lane markings, and every other crucial aspect of a roadway.

That might not seem like such a tough job for the company that gave us Google Earth and Google Maps. But the maps necessary for the Google car are an order of magnitude more complicated. In fact, when I first wrote about the car for MIT Technology Review, Google admitted to me that the process it currently uses to make the maps are too inefficient to work in the country as a whole.

To create them, a dedicated vehicle outfitted with a bank of sensors first makes repeated passes scanning the roadway to be mapped. The data is then downloaded, with every square foot of the landscape pored over by both humans and computers to make sure that all-important real-world objects have been captured. This complete map gets loaded into the car's memory before a journey, and because it knows from the map about the location of many stationary objects, its computer—essentially a generic PC running Ubuntu Linux—can devote more of its energies to tracking moving objects, like other cars.

But the maps have problems, starting with the fact that the car can’t travel a single inch without one. Since maps are one of the engineering foundations of the Google car, before the company's vision for ubiquitous self-driving cars can be realized, all 4 million miles of U.S. public roads will be need to be mapped, plus driveways, off-road trails, and everywhere else you'd ever want to take the car. So far, only a few thousand miles of road have gotten the treatment, most of them around the company's headquarters in Mountain View, California.  The company frequently says that its car has driven more than 700,000 miles safely, but those are the same few thousand mapped miles, driven over and over again.

The common conception of self-driving cars is that they drive somewhat like humans do. That is, they look around at the road and make decisions on what their camera eyes “see.” I didn't realize that the cars were depending so heavily on pre-loaded maps.

I'm still excited about self-driving technology, but my expectations for the near to medium term have become much more modest. I once imagined I could just sit in the backseat of a car and have it drive me to any destination while I futzed around on my phone in the backseat. It's seems clear now that for the foreseeable future, someone always needs to be in the driver's seat, ready to take over at a moment's notice, and the millions of hours of additional leisure team that might be returned to society are not going to materialize this way. We so often throw around self-driving cars as if they're an inevitability, but it may be the last few problems to solve in that space are the most difficult ones to surmount. Thus Moravec's Paradox: it's “difficult or impossible to give [computers] the skills of a one-year-old when it comes to perception and mobility.”

What about an alternative approach, one that pairs humans with computers, a formula for many of the best solutions at this stage in history? What if the car could be driven by a remote driver, almost like a drone?

We live in a country where the government assumes anyone past the age of 16 who passes a driver test can drive for life. Having nearly been run over by many an elderly driver in Palo Alto during my commute to work, I'm not sure that's such a sound assumption. Still, if you are sitting in a car and don't have to drive it yourself, and as long as you get where you need to go, whether a computer does the driving or a remote pilot handles the wheel, you still get that time back.

Adventures in teaching self-driving cars

For complicated moves like that, Thrun’s team often started with machine learning, then reinforced it with rule-based programming—a superego to control the id. They had the car teach itself to read street signs, for instance, but they underscored that knowledge with specific instructions: “stop” means stop. If the car still had trouble, they’d download the sensor data, replay it on the computer, and fine-tune the response. Other times, they’d run simulations based on accidents documented by the National Highway Traffic Safety Administration. A mattress falls from the back of a truck. Should the car swerve to avoid it or plow ahead? How much advance warning does it need? What if a cat runs into the road? A deer? A child? These were moral questions as well as mechanical ones, and engineers had never had to answer them before. The darpa cars didn’t even bother to distinguish between road signs and pedestrians—or “organics,” as engineers sometimes call them. They still thought like machines.

Four-way stops were a good example. Most drivers don’t just sit and wait their turn. They nose into the intersection, nudging ahead while the previous car is still passing through. The Google car didn’t do that. Being a law-abiding robot, it waited until the crossing was completely clear—and promptly lost its place in line. “The nudging is a kind of communication,” Thrun told me. “It tells people that it’s your turn. The same thing with lane changes: if you start to pull into a gap and the driver in that lane moves forward, he’s giving you a clear no. If he pulls back, it’s a yes. The car has to learn that language.”

From Burkhard Bilger's New Yorker piece on Google's self-driving car. The engineering issues they've had to deal with are fascinating.

As many have noted, legal or regulatory risk may be the largest obstacle to seeing self-driving cars on our roads in volume. To counter that, I hypothesize that all self-driving will ship with a black box, like airplanes, and that all the cameras will record a continuous feed of video, that keeps overwriting itself, maybe a loop of the most recent 30 minutes of driving at all times, along with key sensor readings. That way if someone sees the self-driving sensor on a car they can't just back into the self-driving car or hurtle themselves across a windshield just to get a big settlement from Google.

In fact, as sensors and video recording devices come down in cost, it may become law that all cars come with such accessories, self-driving or not, making it much easier to determine fault in car accidents. The same cost/weight improvements in video tech may make it so Amazon drones are also equipped with a continuously recording video camera, the better for determining who may have brought it down with a rock to steal its payload.

Perhaps Google will take the continuous video feeds as a crowd-sourced way to update its street maps. That leads, of course, to the obvious drawback to such a scenario, the privacy concerns over how Google would use the data and video from the cars. That's a cultural issue and seems more tenable than the legal one, however.

Google Autocomplete knows

"Who knows what evil lurks in the hearts of men? The Shadow knows." 

The modern equivalent, one might say, is Google Autocomplete. One ad campaign capitalizes on that by using Google Autocomplete suggestions to highlight gender inequality

google-autocomplete-ad.jpg

More of the images can be seen here

I love Google Autocomplete, it is like some modern crowdsourced, mass-distributed free art installation. I have spent hours putting random queries in there just to plumb the collective mind of humanity.

A surprising corporate giant

What company, according to Fortune, is the eighth largest employer in the world, with over 549,000 employees globally?

The answer shocked me: Volkswagen. That's just the tip of the iceberg in terms of fascinating tidbits from this mini profile.

Efficiency experts will tell you that on an employee-per-vehicle basis, Volkswagen looks hopelessly inefficient. Financial analysts will tell you that the company woefully trails its competitors on a revenue-per-employee basis. But VW will tell you that it makes more money than any other automaker – by far.

While VW's stated goal is to become the world's largest car company by 2018, it's already there if you measure it by revenue and profits. Its revenue of $200 billion is greater than every other OEM. Last year's operating profit of $14 billion is the kind of performance you expect from Big Oil companies, not automakers.

Last year's operating profit of $14 billion is the kind of performance you expect from Big Oil companies, not automakers.

How can this be possible? How can VW look so uncompetitive from a productivity standpoint, yet out-earn all of its competitors?

Ah, that's the magic of VW's corporate structure. While business schools teach future MBAs that centralized operations can cut cost by eliminating overlapping work and duplication, VW maintains strongly decentralized operations with lots of overlap. While business schools preach the benefits of outsourcing to cut cost, VW is very vertically integrated.

Anytime a car company buys a component from a supplier, that supplier has to charge a profit. If an automaker can make those components in-house, it gets to keep that profit. VW is building a lot of components in-house.

To dominate you need multiple brands, and VW has more than anyone else.

If an automaker truly wants to dominate the market, it has to accept a certain amount of overlap and duplication. It just goes with the territory. To dominate you need multiple brands, and VW has more than anyone else, which admittedly overlap at the edges. But to VW they are more than just brands.

All of VW's brands (VW, Audi, Seat, Skoda, Bentley, Lamborghini, Ducati, Porsche, Bugatti, MAN, Scania, and VW Commercial) are treated as stand-alone companies. They have their own boards of directors, their own profit & loss statements, and their own annual reports. They even have their own separate design, engineering and manufacturing facilities. Yes, they do share some platforms and powertrains and purchasing, but other than that they're on their own.

Anyone who works in technology will hear an echo in much of this strategy. Volkswagen's model of of running all its brands as independent companies is an example how the biggest tech companies try to push decision-making to the edges, to the teams running a variety of product lines, as a way of trying to remain entrepreneurial, innovative, and nimble. 

The way Volkswagen has vertically integrated is reminiscent of the way Apple has, over time, taken over more and more of the computing value chain, down to opening their own retail stores. Given how Samsung is also vertically integrating and competing head on with Apple in the mobile computing market, it would be surprising if Apple didn't stop sourcing chips from Samsung and take their business elsewhere, to a partner less vertically integrated, like Taiwan Semiconductor.

Volkswagen, by dint of its vertical integration, can capture value wherever it occurs in the value chain, and as the sources of value shift as it often does over the life cycle of technology products, Volkswagen retains its cut. On a related note, look at the last chart on this post at Asymco. Stunningly, Samsung makes more operating income from Android than Google is! In this mobile computing war, Samsung is making money off of both Google and Apple. After Apple, it's difficult to name another company that has profited more from the mobile computing revolution.

Volkswagen is the answer to the subject of this post, but Samsung is nearly as shocking a dark horse of a corporate behemoth.

Amazon, Apple, and the beauty of low margins

[As always, I preface any discussion of Amazon and Apple by noting that I own some stock in both companies, and that I worked at Amazon from 1997 to 2004]

A lot of folks, especially Apple supporters, like to characterize Amazon as irrational, even crazy, for its willingness to live with low margins. It must be frustrating to compete with a company like that. But to call their strategy irrational or to believe they want to be a non-profit is a dangerous misreading of what they're all about.

It's been years since I worked there, so this is largely speculation on my part, but I believe Amazon is anything but irrational when it comes to how they think about margins. I believe it's a calculated strategy on their part, and anyone competing with them had best understand it.

As with people, I think companies can be more comfortable playing certain styles, much like certain players are more suited for a particular style of offense, like Mike D'Antoni's in the NBA or Chip Kelly's in football. Amazon's low margin strategy is one they are comfortable with because it sprung from the company's very origin. Amazon began in the bookselling business, and some of its earliest and most crucial advantages against incumbents like Barnes and Noble were best expressed with thinner margins.

One of online retail's main advantage was, of course, being able to forego expensive physical storefronts. With one and then two distribution centers total in the early years, Amazon essentially just had two "storefronts" to stock with book SKU's, whereas Barnes and Noble had to guess how to allocate SKU's across hundreds of stores all over the country, all necessitating long leases. A few Amazon editors could recommend books to all Amazon customers, whereas Barnes and Noble had to staff each of their individual stores with sales clerks. 

More importantly, Amazon's inventory flow was drastically more efficient than that of Barnes and Noble. Amazon didn't have to carry inventory on really slow-selling SKU's, they could wait until a customer had ordered it and then drop-ship it from the distributor. If Amazon wanted to ship one of those SKU's themselves, customers generally had the patience to wait longer for them since those slow-turning SKU's didn't earn shelf space at the local Barnes and Noble anyway.

Almost all customers paid by credit card, so Amazon would receive payment in a day. But they didn't pay the average distributor or publisher for 90 days for books they purchased. This gave Amazon a magical financial quality called a negative operating cycle. With every book sale, Amazon got cash it could hang on to for up weeks on end (in practice it wasn't actually 89 days of float since Amazon did purchase some high velocity selling books ahead of time). The more Amazon grew, the more cash it banked. Amazon was turning its inventory 30, 40 times a year, whereas companies like Barnes and Noble were sweating to turn their inventory twice a year. Most people just look at a company's margins and judge the quality of the business model based on that, but the cash flow characteristics of the business can make one company a far more valuable company than another with the exact same operating margin. Amazon could have had a margin of zero and still made money.

At Amazon we were ruthlessly focused on squeezing inefficiency out of every part of the business, especially the variable ones that affected every purchase. How could we get a book from the shelf into the hands of the customer more cheaply? How could we reduce the number of customer contacts per order for our customer service team? Could we offload some human customer service contact to cheaper online self-service methods while improving customer satisfaction? How could we negotiate steeper discounts on the books themselves? For each book SKU, was it more economical to purchase ahead of sales in bulk for steeper discounts and faster shipping or to purchase only when a customer placed an order and risk a longer delay in shipping? How could we allocate inventory among our distribution centers to increase the likelihood that all items in an order shipped from the same distribution center, minimizing our shipping costs? How could we organize all the Amazon shipments ready for delivery in a way that made lives easier on our shipping partners like the USPS and UPS, and then how could we use that to negotiate cheaper shipping rates? Did we need so many human editors reviewing books, or were customer reviews sufficient?

The type of operational efficiency Amazon rose to in those days is not something another company can duplicate overnight. It came on top of the inherent cost advantages of online commerce over physical commerce. So much of Amazon's competitive advantage in those days came from operational efficiency. You can choose to leverage that strength in two ways. One is you match your competitor on pricing and just earn higher margins. But the other, the way Amazon has always tended to favor, is to lower prices, to thin the oxygen for your competitors.

If you have bigger lungs than your competitor, all things being equal, force them to compete in a contest where oxygen is the crucial limiter. If your opponent can't swim, you make them compete in water. If they dislike the cold, set the contest in the winter, on a tundra. You can romanticize all of this by quoting Sun Tzu, but it's just common sense.

I worked on the launch of the Amazon Video store, Amazon's third product after books and music. At the time of the launch, DVDs had just launched as a product category a short while earlier, so the store carried both VHS tapes and DVDs. The day Amazon launched its video store, the top DVD store on the web at the time, I think it was DVD Empire, lowered its prices across the board, raising its average discount from 30% off to 50% off DVDs.

This forced our hand immediately. Selling DVDs at 50% off would mean selling those titles at a loss. We had planned to match their 30% discount, and now we were being out-priced by the market leader on our first day of operation, and just before the heart of the holiday sales season to boot (it was November, 1998).

We convened a quick emergency huddle, but it didn't take long to come to a decision. We'd match the 50% off. We had to. Our leading opponent had challenged us to a game of who can hold your breath longer. We were confident in our lung capacity. They only sold DVDs whereas we had the security of a giant books and music business buttressing our revenues.

After a few weeks, DVD Empire blinked. They had to. Sometime later, I can't remember how long it was, DVD Empire rebranded, tried expanding to sell adult DVDs, then went out of business. There were other DVD-only retailers online at the time, but none from that period survived. I doubt any online retailer selling only DVDs still exists.

Attacking the market with a low margin strategy has other benefits, though, ones often overlooked or undervalued. For one thing, it strongly deters others from entering your market. Study disruption in most businesses and it almost always comes from the low end. Some competitor grabs a foothold on the bottom rung of the ladder and pulls itself upstream. But if you're already sitting on that lowest rung as the incumbent, it's tough for a disruptor to cling to anything to gain traction.

An incumbent with high margins, especially in technology, is like a deer that wears a bullseye on its flank. Assuming a company doesn't have a monopoly, its high margin structure screams for a competitor to come in and compete on price, if nothing else, and it also hints at potential complacency. If the company is public, how willing will they be to lower their own margins and take a beating on their public valuation?

Because technology, both hardware and software, tends to operate on an annual update cycle, every year you have to worry about a competitor leapfrogging you in that cycle. One mistake and you can see a huge shift in customers to a competitor.

Not having to sweat a constant onslaught of new competitors is really underrated. You can allocate your best employees to explore new lines of business, you can count on a consistent flow of cash from your more mature product or service lines, and you can focus your management team on offense. In contrast, most technology companies live in constant fear that they'll be disrupted with every product or service refresh. The slightest misstep can turn a stock market darling into a company struggling for its very existence.

Amazon's core retail business is, I'd argue, still very secure. I can't think of a tech retail competitor that is a legitimate threat to Amazon in selling most physical goods. Where Amazon is most vulnerable in retail is those areas where the game shifted on them, and that's in the media lines where physical books, CDs, and DVDs are being digitized. Since no physical product must be transported through a distribution system, Amazon's operational efficiency advantages there are less effective against competition. But in the arena of buying something online and having a box delivered to your doorstep, who really scares Amazon?

Another advantage to low margin models is increased customer loyalty. Most of the products Amazon sells are commodity items. It's not like buying one brand of car versus another, where you a variety of subjective judgements affect the consumer's choice. The Avengers Blu-ray disc you buy from Amazon is the same one you'll find at Wal-Mart or Best Buy. In that world, the lowest price tends to win. In the early years, Amazon routinely lowered either product pricing or shipping pricing. Very few companies lower their prices permanently as time goes by except on depreciating goods, like computers whose value decreases as newer, faster models hit the market.

If you're the low-cost leader, customers will forgive a lot of sins. That margin of error, like the competitive moat, buys you peace of mind. I could spend time price-shopping every item on Amazon, but these days, I don't really bother. Amazon's website design is not going to win any design awards, it's a bit of a Frankensteinian assemblage thanks to distributed design decisions, but it's fast, the shipping is cheap or free, the customer service is fantastic, and oh, did I mention, their prices are great! There is value in being the site of first and last resort for customers.

If you want to jump into competition with Amazon, you can't just match Amazon, you have to leapfrog them. But they've left almost no price umbrella for you to sneak under, so you have to both match them in price and then blow them away on the user experience side to even get customers to think about switching. Who has the capital and wherewithal to play that exceedingly unpleasant, unprofitable game? You can only win that game at scale, and Amazon already achieved it.

Smart companies compete first by playing to their strengths, but Amazon also cleaves to a low margin strategy, I believe, because it's demonstrated the advantages noted above. Amazon could try to build a high margin tablet to compete with Apple, but why would they? How have companies that have tried to challenge Apple with design and build quality fared these past few years? Why would you try to challenge Apple in the areas it is strongest at?

In a recent interview, Reed Hastings claimed Amazon was spending $1 billion a year on licensing streaming video for Amazon Instant Video. Hastings is negotiating for much of the same content, I know he knows what that content costs, and since I used to work at Hulu, I can vouch for how easy it would be to burn through a billion dollars building up a substantial streaming video library. I do think Amazon may have overpaid as a consequence of wanting to come in strong and make a big play without as much pricing information as Netflix and Hulu have accumulated over the years, but it strikes me as a classic tactic out of the Amazon low end disruption playbook.

[In this world of digital video, this strategy is much more difficult to execute because there is no fixed price on licensing episodes of TV shows and libraries of movies. The information asymmetry works in favor of the content providers. Netflix had a great advantage when First Sale Doctrine permitted them to buy DVDs at the same wholesale price as any retailer since it capped their costs. But in the TV/movie licensing world, the content owner can constantly adjust their price to squeeze almost every last drop of margin from the distributor as you can't find perfect substitutes for the goods being offered. Ask TV networks if they make any money licensing NFL, NBA, and MLB games for broadcast. Hint: the answer is no. Ask companies like Apple and Spotify if they're making healthy margins selling digital music. Ask Netflix or Amazon if licensing TV shows and movies for digital streaming is more or less profitable than the days of selling or renting physical media. In the digital world, transfer pricing can be even more of a cruel mistress. 

Most companies building profitable ecosystems in the digital world are making their profits elsewhere using the digital media as a loss leader. Apple on its hardware, for example, or TV networks trying to use sports contests to cross-promote their other TV programs.]

Apple took some grief last quarter for seeing some margin depression, but in and of itself, I don't see that as a bad sign. In fact, I was disappointed that Apple didn't price the iPad Mini lower out of the gate. Of course, they're largely sold out through the holidays, so pricing it lower means leaving money on the table in the conventional microeconomic analysis.

But in the long run, if you look at every iPad purchaser as a new subscriber to the Apple ecosystem of hardware and software services, there's value in fighting for every additional user versus Google or Amazon in the low end tablet market. Most customers who buy a low end tablet will stay in that producer's ecosystem for a while, at least a year. Graph the low end market and you see it trending towards zero, to that day when an Amazon or a Google will likely offer you a low end tablet for free, perhaps as part of your Amazon Prime subscription or if you agree to pay for Google Drive.

That's a world in which the switching costs are set by the software ecosystem of each of those companies, not the hardware. It's why Apple lovers are right to fret about iCloud and its underwhelming mail, storage, and calendaring services and substandard reliability, why Amazon might spend a billion dollars licensing videos, why Google tried so hard to switch people over to Google+. They're all looking for a path to software lockin, a more defensible moat.

Apple still is the margin king among those competitors in the mobile phone and tablet spaces in which they compete. But if they decided to start using their low-end priced SKU's in mobile phones and tablets to press down on Google and Amazon, and if their margins declined as a result, I, as a shareholder, wouldn't necessarily find that to be a negative. I would love to find the sales mix data on their different SKU's in the iPhone and iPad verticals, though I have yet to see that data shared publicly anywhere. The shape of that curve will tell us a lot about where those markets are in their lifecycle, but Apple has some control over their shape as well.

Some might say that Apple doesn't have the right mindset to play low-margin offense, that it's against their nature. But they've effectively dominated and wrung every last drop of money from the iPod market using pieces of this strategy, and they have the operational expertise and vertical integration to achieve it. In fact, Apple now turns its inventory more times a year than Amazon, by a healthy margin, a staggering fact.

I haven't mentioned Google much, but like Amazon they will continue to attack Apple at the low end with their strategy of subsidizing businesses with their core ad revenue. For the forseeable future, Apple will have these two giants snatching at their feet. It's a high pressure, high stakes game. Wouldn't it be nice to trade some margin for higher castle walls, just for peace of mind? 

Most people don't appreciate them, but low margins have their own particular brand of beauty.