Why the world is getting weirder

It used to be that airliners broke up in the sky because of small cracks in the window frames. So we fixed that. It used to be that aircraft crashed because of outward opening doors. So we fixed that. Aircraft used to fall out of the sky from urine corrosion, so we fixed that with encapsulated plastic lavatories. The list goes on and on. And we fixed them all.

So what are we left with?

As we find more rules to fix more things we are encountering tail events. We fixed all the main reasons aircraft crash a long time ago. Sometimes a long, long time ago. So, we are left with the less and less probable events.

We invented the checklist. That alone probably fixed 80% of fatalities in aircraft. We’ve been hammering away at the remaining 20% for 50 years or so by creating more and more rules.

We’ve reached the end of the useful life of that strategy and have hit severely diminishing returns. As illustration, we created rules to make sure people can’t get in to cockpits to kill the pilots and fly the plane in to buildings. That looked like a good rule. But, it’s created the downside that pilots can now lock out their colleagues and fly it in to a mountain instead.

From a great piece by Steve Coast on why the world is getting weirder. Follow the Pareto Principle long enough and you fix all the low-hanging fruit with a whole bunch of rules, leaving just the black swans unaccounted for.

Anyone who has worked on any tech product or service long enough, through many cycles, knows you can end up working on just edge cases. Often, when you hit this point, you're listening to a sliver of power users and are at the point of such diminishing returns that accommodating them might be counterproductive as a whole. All you do by adding that random feature they want is add some interface overhead and friction for majority of your users, whose problems you already solved.

At this point, if the user base is large and healthy enough, most smart and ambitious companies move on to launching new products and services with higher marginal returns on their resources ³ . The resource vacuum is often exacerbated by the fact that the most ambitious employees would rather work on the new new thing. So they move on to the latest hot top secret project, leaving the former product or service in a maintenance mode, with minimal oversight.

1. Large and successful multi-product companies that reach this point often just kill off the product or service if the user base isn't large enough. Think Google Reader or Apple's Ping. A startup that reaches that point often pivots, sells themselves, or folds. ↩

It's usually the right near-term economic thing to do, but it can also leave some widely used products or services with chronic issues or imperfections that puzzle users and outsiders. How, they wonder, can a company with thousands of employees not bother to fix such longstanding and seemingly trivial issues? This is why competition is healthy, even if sometimes it seems like we have too many redundant products/services in tech.

Coast's post also includes some good career advice.

On a personal level we should probably work in areas where there are few rules.

To paraphrase Peter Thiel, new technology is probably so fertile and productive simply because there are so few rules. It’s essentially illegal for you to build anything physical these days from a toothbrush (FDA regulates that) to a skyscraper, but there’s zero restriction on creating a website. Hence, that’s where all the value is today.

If we can measure economic value as a function of transactional volume (the velocity of money for example), which appears reasonable, then fewer rules will mean more volume, which means better economics for everyone.

“F***, we’re dead.” [UPDATED]

On the last day of May in 2009, as night enveloped the airport in Rio de Janeiro, the 216 passengers waiting to board a flight to Paris could not have suspected that they would never see daylight again, or that many would sit strapped to their seats for another two years before being found dead in the darkness, 13,000 feet below the surface of the Atlantic Ocean. But that is what happened. Air France Flight 447 carried a crew of nine flight attendants and three pilots—their numbers augmented because of duty-time limitations on a 5,700-mile trip that was expected to last nearly 11 hours. These were highly trained people, flying an immaculate wide-bodied Airbus A330 for one of the premier airlines of the world, an iconic company of which all of France is proud. Even today—with the flight recorders recovered from the sea floor, French technical reports in hand, and exhaustive inquests under way in French courts—it remains almost unimaginable that the airplane crashed. A small glitch took Flight 447 down, a brief loss of airspeed indications—the merest blip of an information problem during steady straight-and-level flight. It seems absurd, but the pilots were overwhelmed.

An absolutely riveting read about the crash of Air France Flight 447 in 2009. I'm simultaneously devastated and enthralled by air flight disaster stories.

Much ink is spilled about self-driving cars, but in the meantime we've shifted into a self-flying world, and that means the airplane crashes that do occur are at the intersection of an old model of flying, completely human dependent, and a newer model of flying that is heavily reliant on computer flying. Also worth reading for an example of an area where user interface design is a matter of literal life and death, unlike that of most of what those of us in technology dabble in. 

Despite all that, it's worth remembering that flying has never been safer.

The events of the past several months, punctuated by the losses of Malaysia Airlines flights 370 and 17, have given many people the idea that flying has become less safe. In fact it’s much safer than it used to be. There are twice as many planes in the air as there were 25 years ago, yet the rate of fatal accidents, per miles flown, has been steadily falling. The International Civil Aviation Organization reports that for every million flights, the chance of a crash is one-sixth what it was in 1980.

Globally, 2013 was the safest year in the history of modern commercial aviation. This year will be something of a correction, but we can’t expect every year to be the safest, and the overall trend shouldn’t be affected. If you think the past 12 months have been bad, go back to 1985, when 27 (!) serious aviation accidents killed almost 2,500 people. Two of history’s ten deadliest disasters happened that year, within two months of each other. The 60s, 70s and 80s were an era rife with horrific crashes, bombings, airport attacks and so on. Recent events notwithstanding, large-scale disasters have become a lot less frequent.

UPDATE (12 Nov 2014): Patrick Smith of Ask the Pilot fame disagrees with the author of the piece, William Langewiesche, that increased automation and declining human piloting skills were the key culprit in the crash.

As Langewiesche has it, the piloting profession doesn’t amount to much. At one point he writes of pilots: “All of them think they are better than they are.” I wonder if he’d make such a rude and cursory blanket statement about doctors or other professionals.

At that, at least, I was able to laugh out loud. The point where I had steam coming from my ears came a few pages later: “In professional flying, a historical shift has occurred,” writes Langewiesche near the end of the piece. “Pilots have been relegated to mundane roles as system managers, expected to monitor the computers and sometimes to enter data via keyboards, but to keep their hands off the controls, and to intervene only in the rare even to failure.”

That is about the most asinine and misleading characterization of an airline pilot’s job that I have ever read in my life.

Smith includes Langewiesche's response at the bottom of the post.

Asiana Flight 214

Patrick Smith of Ask the Pilot  fame provided an informed critique of many of the hasty judgments people grasped at wildly after Asiana Flight 214 crashed at SFO. Among those:

Lastly, we're hearing murmurs already about the fact that Asiana Airlines hails from Korea, a country with a checkered past when it comes to air safety. Let's nip this storyline in the bud. In the 1980s and 1990s, that country's largest carrier, Korean Air, suffered a spate of fatal accidents, culminating with the crash of Flight 801 in Guam in 1997. The airline was faulted for poor training standards and a rigid, authoritarian cockpit culture. The carrier was ostracized by many in the global aviation community, including its airline code-share partners. But Korean aviation is very different today, following a systemic and very expensive overhaul of the nation’s civil aviation system. A 2008 assessment by ICAO, the civil aviation branch of the United Nations, ranked Korea's aviation safety standards, including its pilot training standards, as nothing less than the highest in the world, beating out more than 100 other countries. As they should be, Koreans are immensely proud of this turnaround, and Asiana Airlines, the nation's No. 2 carrier, had maintained an impeccable record of both customer satisfaction and safety.

Whatever happened on final approach into SFO, I highly doubt that it was anything related to the culture of Korean air safety in 2013. Plane crashes are increasingly rare the world over. But they will continue to happen from time to time, and no airline or country is 100 percent immune. 

Malcolm Gladwell arguably did more than anyone to popularize the theory that Korean culture lay at the root of Korean Air's poor safety record in the 1980's and 1990's. In this interview with CNN Money, he summarized his theory from his book Outliers:

F: You share a fascinating story about culture and airline safety.

G: Korean Air had more plane crashes than almost any other airline in the world for a period at the end of the 1990s. When we think of airline crashes, we think, Oh, they must have had old planes. They must have had badly trained pilots. No. What they were struggling with was a cultural legacy, that Korean culture is hierarchical. You are obliged to be deferential toward your elders and superiors in a way that would be unimaginable in the U.S.

But Boeing (BA, Fortune 500) and Airbus design modern, complex airplanes to be flown by two equals. That works beautifully in low-power-distance cultures [like the U.S., where hierarchies aren't as relevant]. But in cultures that have high power distance, it's very difficult.

I use the case study of a very famous plane crash in Guam of Korean Air. They're flying along, and they run into a little bit of trouble, the weather's bad. The pilot makes an error, and the co-pilot doesn't correct him. But once Korean Air figured out that their problem was cultural, they fixed it.

A fairly thorough rebuttal to Gladwell's theory was posted at the blog Ask a Korean:

First, the way in which Gladwell quoted the transcript is severely misleading. This is the full transcript, which goes from pp. 185 to 187 of the NTSB report:

CAPTAIN: 어... 정말로... 졸려서... (불분명) [eh... really... sleepy... (unintelligible words)]

FIRST OFFICER: 그럼요 [Of course]

FIRST OFFICER: 괌이 안 좋네요 기장님 [Captain, Guam condition is no good]

FIRST OFFICER: Two nine eighty-six

CAPTAIN: 야! 비가 많이 온다 [Uh, it rains a lot]

CAPTAIN: (unintelligible words)

CAPTAIN: 가다가 이쯤에서 한 20 마일 요청해 [Request twenty miles deviation later on]

FIRST OFFICER: 네 [yes]

CAPTAIN: ... 내려가면서 좌측으로 [... to the left as we are descending]

(UNCLEAR SPEAKER): (chuckling, unintelligible words)

FIRST OFFICER: 더 오는 것같죠? 이 안에. [Don't you think it rains more? In this area, here?]

(emphases mine)

Note the difference between the full transcript, and the way Gladwell presented the transcript. Gladwell only quoted the first two lines and the last line of this sequence, omitting many critical lines in the process. In doing so, Gladwell wants to create an impression that the first officer underwent some period of silent contemplation, and decided to warn the captain of the poor weather conditions in an indirect, suggestive manner. 

The full transcript reveals that this is clearly not the case. The first officer spoke up directly, clearly, and unmistakably:  "Captain, Guam condition is no good." It is difficult to imagine how a person could be more direct about the poor weather condition. Further, there was no silent contemplation by the first officer. Nearly three minutes elapse during this sequence, during the captain and the first officer chatted constantly. And it is the captain who first brings up the fact that it is raining a great deal: "Uh, it rains a lot." In this context, it is clear that the first officer is engaged in some friendly banter about the rain, not some indirect, ominous warning about the flight conditions.

To be fair to Gladwell, when asked about whether he thought his theory from Outliers came into play in the case of Asiana Flight 214, he did not bite. 

We asked Malcolm Gladwell for his thoughts on the use of his essay in the particular context of the Asiana crash. "I can understand why my Outliers chapter has been of interest, given how central cockpit communication issues are in plane crashes," Gladwell told The Atlantic Wire in an email, adding, "My sense is that we should wait for the full report on the crash before drawing any conclusions about its cause." As for the applicability of his work to the recent Asiana crash, Gladwell noted that his essay was specific to the problems (and solutions) of one airline — Korean Air, "which I think did an extraordinary job of addressing the cultural issues involved in pilot communication. This was a crash involving a completely different airline," he said. 

The NTSB has yet to issue any formal assessment of what happened that day. I happened to arrive at SFO for a flight to Paris just a few hours after the crash occurred, and we could see the wreckage in the distance from our gate. Our flight was delayed by 9 hours, and we finally took off at around 9pm that night.

Our plane turned onto the runway and accelerated towards takeoff. The moment just before our wheels left the ground, I saw, just out the window to the left, the wreckage of Asiana Flight 214, sitting just off our runway, illuminated by some giant spotlights, like a giant burned out metallic skeleton that had just been dug out of the earth.

Replacing black box recorders

It's probably some combination of steel and titanium, but the exact composition is a secret, as are the ingredients in the wafer of insulating material that preserves the data on the chip even after it's cooked for an hour at over 2,000°C. In their rites of passage, black boxes are shot out of a cannon at a crushing 3,400 times the force of Earth's gravity, squeezed in a hydraulic press at 2,000 kilograms of pressure for five minutes and subjected to water pressure at a simulated depth of 6,000 metres. They are soaked in jet fuel, lubricating oil and hydraulic fluid for 48 hours, and immersed in seawater for 30 days. The housing isn't meant to be watertight -- you don't want 600 atmospheres of pressure differential bearing on the walls, Schmutz says -- but the data has to survive anyway.

Black box flight recorders are miraculous things, able to preserve the data on its memory chips even under the most trying physical and chemical distress (as one comedian, I can't remember which, once joked, shouldn't we make planes out of the same material we make these black boxes from). But wouldn't it just be easier to stream that data to land so we wouldn't have to track down the flight recorders after plane accidents?