Remove the legend to become one

When I started my first job at Amazon.com, as the first analyst in the strategic planning department, I inherited the work of producing the Analytics Package. I capitalize the term because it was both a serious tool for making our business legible, and because the job of its production each month ruled my life for over a year.

Back in 1997, analytics wasn't even a real word. I know because I tried to look up the term, hoping to clarify just I was meant to be doing, and I couldn't find it, not in the dictionary, not on the internet. You can age yourself by the volume of search results the average search engine returned when you first began using the internet in force. I remember when pockets of wisdom were hidden in eclectic newsgroups, when Yahoo organized a directory of the web by hand, and later when many Google searches returned very little, if not nothing. Back then, if Russians wanted to hack an election, they might have planted some stories somewhere in rec.arts.comics and radicalized a few nerds, but that's about it.

Though I couldn't find a definition of the word, it wasn't difficult to guess what it was. Some noun form of analysis. More than that, the Analytics Package itself was self-describing. Literally. It came with a single page cover letter, always with a short preamble describing its purpose, and then jumped into a textual summary of the information within, almost like a research paper abstract, or a Letter to Shareholders. I like to think Jeff Bezos' famous company policy, instituted many years later, banning Powerpoint in favor of written essays, had some origins in the Analytics Package cover letter way back when. The animating idea was the same: if you can't explain something in writing to another human, do you really understand it yourself?

My interview loop at Amazon ended with an hour with the head of recruiting at the time, Ryan Sawyer. After having gone through a gauntlet of interviews that included almost all the senior executives, and including people like Jeff Bezos and Joy Covey, some of the most brilliant people I've ever met in my life, I thought perhaps the requisite HR interview would be a letup. But then Ryan asked me to explain the most complex thing I understood in a way he'd understand. It would be good preparation for my job.

What was within the Analytics Package, that required a written explanation? Graphs. Page after page of graphs, on every aspect of Amazon's business. Revenue. Editorial. Marketing. Operations. Customer Service. Headcount. G&A. Customer sentiment. Market penetration. Lifetime value of a customer. Inventory turns. Usually four graphs to a page, laid out landscape.

The word Package might seem redundant if Analytics is itself a noun. But if you saw one of these, you knew why it was called a Package. When I started at Amazon in 1997, the Analytics Package was maybe thirty to forty pages of graphs. When I moved over to product management, over a year later, it was pushing a hundred pages, and I was working on a supplemental report on customer order trends in addition. Analytics might refer to a deliverable or the practice of analysis, but the Analytics Package was like the phone book, or the Restoration Hardware catalog, in its heft.

This was back in the days before entire companies focused on building internal dashboards and analytical tools, so the Analytics Package was done with what we might today consider as comparable to twigs and dirt in sophistication. I entered the data by hand into Excel tables, generated and laid out the charts in Excel, and printed the paper copies.

One of the worst parts of the whole endeavor was getting the page numbers in the entire package correct. Behind the Analytics Package was a whole folder of linked spreadsheets. Since different charts came from different workbooks, I had to print out an entire Analytics Package, get the ordering correct, then insert page numbers by hand in some obscure print settings menu. Needless to say, ensuring page breaks landed where you wanted them was like defusing a bomb.

Nowadays, companies hang flat screen TVs hanging on the walls, all them running 24/7 to display a variety of charts. Most everyone ignores them. The spirit is right, to be transparent all the time, but the understanding of human nature is not. We ignore things that are shown to us all the time. However, if once a month, a huge packet of charts dropped on your desk, with a cover letter summarizing the results, and if the CEO and your peers received the same package the same day, and that piece of work included charts on how your part of the business was running, you damn well paid attention, like any person turning to the index of a book on their company to see if they were mentioned. Ritual matters.

The package went to senior managers around the company. At first that was defined by your official level in the hierarchy, though, as most such things go, it became a source of monthly contention as to who to add to the distribution. One might suspect this went to my head, owning the distribution list, but in fact I only cared because I had to print and collate the physical copies every month.

I rarely use copy machines these days, but that year of my life I used them more than I will all the days that came before and all the days still to come, and so I can say with some confidence that they are among the least reliable machines ever made by mankind.

It was a game, one whose only goal was to minimize pain. A hundred copies of a hundred page document. The machine will break down at some point. A sheet will jam somewhere. The ink cartridge will go dry. How many collated copies do you risk printing at once? Too few and you have to go through the setup process again. Too many and you risk a mid-job error, which then might cascade into a series of ever more complex tasks, like trying to collate just the pages still remaining and then merging them with the pages that were already completed. [If you wondered why I had to insert page numbers by hand, it wasn't just for ease of referencing particular graphs in discussion; it was also so I could figure out which pages were missing from which copies when the copy machine crapped out.]

You could try just resuming the task after clearing the paper jam, but in practice it never really worked. I learned that copy machine jams on jobs of this magnitude were, for all practical purposes, failures from which the machine could not recover.

I became a shaman to all the copy machines in our headquarters at the Columbia building. I knew which ones were capable of this heavy duty task, how reliable each one was. Each machine's reliability fluctuated through some elusive alchemy of time and usage and date of the last service visit. Since I generally worked late into every night, I'd save the mass copy tasks for the end of my day, when I had the run of all the building's copy machines.

Sometimes I could sense a paper jam coming just by the sound of machine's internal rollers and gears. An unhealthy machine would wheeze, like a smoker, and sometimes I'd put my hands on a machine as it performed its service for me, like a healer laying hands on a sick patient. I would call myself a copy machine whisperer, but when I addressed them it was always a slew of expletives, never whispered. Late in my tenure as analyst, I got budget to hire a temp to help with the actual printing of the monthly Analytics Package, and we keep in touch to this date, bonded by having endured that Sisyphean labor.

My other source of grief was another tool of deep fragility: linked spreadsheets in Excel 97. I am, to this day, an advocate for Excel, the best tool in the Microsoft Office suite, and still, if you're doing serious work, the top spreadsheet on the planet. However, I'll never forget the nightmare of linked workbooks in Excel 97, an idea which sounded so promising in theory and worked so inconsistently in practice.

Why not just use one giant workbook? Various departments had to submit data for different graphs, and back then it was a complete mess to have multiple people work in the same Excel spreadsheet simultaneously. Figuring out whose changes stuck, that whole process of diffs, was untenable. So I created Excel workbooks for all the different departments. Some of the data I'd collect myself and enter by hand, while some departments had younger employees with the time and wherewithal to enter and maintain the data for their organization.

Even with that process, much could go wrong. While I tried to create guardrails to preserve formulas linking all the workbooks, everything from locked cells to bold and colorful formatting to indicate editable cells, no spreadsheet survives engagement with a casual user. Someone might insert a column here or a row there, or delete a formula by mistake. One month, a user might rename a sheet, or decide to add a summary column by quarter where none had existed before. Suddenly a slew of #ERROR's show up in cells all over the place, or if you're unlucky, the figures remain, but they're wrong and you don't realize it.

Thus some part of every month was going through each spreadsheet and fixing all the links and pointers, reconnecting charts that were searching for a table that was no longer there, or more insidiously, that were pointing to the wrong area of the right table.

Even after all that was done, though, sometimes the cells would not calculate correctly. This should have been deterministic. That's the whole idea of a spreadsheet, that the only error should be user error. A cell in my master workbook would point at a cell in another workbook. They should match in value. Yet, when I opened both workbooks up, one would display 1,345 while the other would display 1,298. The button to force a recalculation of every cell was F9. I'd press it repeatedly. Sometimes that would do it. Sometimes it wouldn't. Sometimes I'd try Ctrl - Alt - Shift - F9. Sometimes I'd pray.

One of the only times I cried at work was late one night, a short time after my mom had passed away from cancer, my left leg in a cast from an ACL/MCL rupture, when I could not understand why my workbooks weren't checking out, and I lost the will, for a moment, to wrestle it and the universe into submission. This wasn't a circular reference, which I knew could be fixed once I pursued it to the ends of the earth, or at least the bounds of the workbook. No, this inherent fragility in linked workbooks in Excel 97 was a random flaw in a godless program, and I felt I was likely the person in the entire universe most fated to suffer its arbitrary punishment.

I wanted to leave the office, but I was too tired to go far on my crutches. No one was around the that section of the office at at that hour. I turned off the computer, turned out the lights, put my head down on my desk for a while until the moment passed. Then I booted the PC back up, opened the two workbooks, and looked at the two cells in question. They still differed. I pressed F9. They matched. 

Most months, after I had finished collating all the copies of the Analytics Package, clipping each with a small, then later medium, and finally a large binder clip, I'd deliver most copies by hand, dropping them on each recipient's empty desk late at night. It was a welcome break to get up from my desk and stroll through the offices, maybe stop to chat with whoever was burning the midnight oil. I felt like a paper boy on his route, and often we'd be up at the same hour.

For all the painful memories that cling to the Analytics Package, I consider it one of the formative experiences of my career. In producing it, I felt the entire organism of our business laid bare before me, its complexity and inner working made legible. The same way I imagine programmers visualizing data moving through tables in three dimensional space, I could trace the entire ripple out from a customer's desire to purchase a book, how a dollar of cash flowed through the entire anatomy of our business. I knew the salary of every employee, and could sense the cost of their time from each order as the book worked its way from a distributor to our warehouse, from a shelf to a conveyor belt, into a box, then into a delivery truck. I could predict, like a blackjack player counting cards in the shoe, what % of customers from every hundred orders would reach out to us with an issue, and what % of those would be about what types of issues.

I knew, if we gained a customer one month, how many of their friends and family would become new customers the next month, through word of mouth. I knew if a hundred customers made their first order in January of 1998, what % of them would order again in February, and March, and so on, and what the average basket size of each order would be. As we grew, and as we gained some leverage, I could see the impact on our cash flow from negotiating longer payable days with publishers and distributors, and I'd see our gross margins inch upwards every time we negotiated better discounts off of list prices.

What comfort to live in the realm of frequent transactions and normal distributions, a realm where the laws of large numbers was the rule of law. Observing the consistency and predictability of human purchases of books (and later CDs and DVDs) each month was like spotting some crystal structure in Nature under a microscope. I don't envy companies like Snapchat or Twitter or Pinterest, social networks who have gone public or likely have to someday, companies who play in the social network business, trying to manage investor expectations when their businesses are so large and yet still so volatile, their revenue streams even more so. It is fun to grow with the exponential trajectory of a social network, but not fun if you're Twitter trying to explain every quarter why you missed numbers again, and less fun when you have to pretend to know what will happen to revenue one quarter out, let alone two or three.

At Amazon, I could see our revenue next quarter to within a few percentage points of accuracy, and beyond. The only decision was how much to tell Wall Street we anticipated our revenue being. Back then, we always underpromised on revenue; we knew we'd overdeliver, the only question was how much we should do so and still maintain a credible sense of surprise on the next earnings call.

The depth of our knowledge of our own business continues to exceed that of any company I've worked at since. Much of the credit goes to Jeff for demanding that level of detail. No one can set a standard for accountability like the person at the top. Much credit goes to Joy and my manager Keith for making the Analytics Package one of the strategic planning department's central tasks. That Keith pushed me into the arms of Tufte changed everything. And still more credit belongs to all the people who helped gather obscure bits of data from all parts of the business, from my colleagues in accounting to those in every department in the company, many of whom built their own models for their own areas, maintaining and iterating them with a regular cadence because they knew every month I'd come knocking and asking questions.

I'm convinced that because Joy knew every part of our business as well or better than almost anyone running them, she was one of those rare CFO's that can play offense in addition to defense. Almost every other CFO I've met hews close to the stereotype; always reigning in spending, urging more fiscal conservatism, casting a skeptical eye on any bold financial transactions. Joy could do that better than the next CFO, but when appropriate she would urge us to spend more with a zeal that matched Jeff's. She, like many visionary CEO's, knew that sometimes the best defense is offense, especially when it comes to internet markets, with their pockets of winner-take-all contests, first mover advantages, and network effects.

It still surprises me how many companies don't help their employees understand the numeric workings of their business. One goes through orientation and hears about culture, travel policies, where the supply cabinet is, maybe some discussion of mission statements. All valuable, of course. But when was the last time any orientation featured any graphs on the business? Is it that we don't trust the numeracy of our employees? Do we fear that level of radical transparency will overwhelm them? Or perhaps it's a mechanism of control, a sort of "don't worry your little mind about the numbers" and just focus on your piece of the puzzle?

Knowing the numbers isn't enough in and of itself, but as books like Moneyball make clear, doing so can reveal hidden truths, unknown vectors of value (for example, in the case of Billy Beane and the Oakland A's, on base percentage). To this day, people still commonly talk about Amazon not being able to turn a profit for so many years as if it is some Ponzi scheme. Late one night in 1997, a few days after I had started, and about my third or fourth time reading the most recent edition of the Analytics Package cover to back, I knew our hidden truth: all the naysaying about Amazon's profitless business model was a lie. Every dollar of our profit we didn't reinvest into the business, and every dollar we didn't raise from investors to add to that investment, would be just kneecapping ourselves. The only governor of our potential was the breadth of our ambition.

***

What does this have to do with line graphs? A month or two into my job, my manager sent me to a seminar that passed through Seattle. It was a full day course centered around the wisdom in one book, taught by the author. The book was The Visual Display of Quantitative Information, a cult bestseller on Amazon.com, the type of long tail book that, in the age before Amazon, might have remained some niche reference book, and the author was Edward Tufte. It's difficult to conjure, on demand, a full list of the most important books I've read, but this is one.

My manager sent me to the seminar so I could apply the principles of that book to the charts in the Analytics Package. My copy of the book sits on my shelf at home, and it's the book I recommend most to work colleagues.

In contrast to this post, which has buried the lede so far you may never find it, Tufte's book opens with a concise summary of its key principles.

Excellence in statistical graphics consists of complex ideas communicated with clarity, precision, and efficiency. Graphics displays should

  • show the data
  • induce the viewer to think about the substance rather than about methodology, graphic design, the technology of graphic production, or something else
  • avoid distorting what the data have to say
  • present many numbers in a small space
  • make large data sets coherent
  • encourage the eye to compare different pieces of data
  • reveal the data at several levels of detail, from a broad overview to the fine structure
  • serve a reasonably clear purpose: description, exploration, tabulation, or decoration
  • be closely integrated with the statistical and verbal descriptions of a data set.

Graphics reveal data. Indeed graphics can be more precise and revealing than conventional statistical computations.

That's it. The rest of the book is just one beautiful elaboration after another of those first principles. The world in one page.

Of all the graphs, the line graph is the greatest. Of its many forms, the most iconic form, the one I used the most in the Analytics Package, has time as the x-axis and the dimension to be measured as the y-axis. Data trended across time.

One data point is one data point. Two data points, trended across time, tell a story. [I'm joking, please don't tell a story using just two data points] The line graph tells us where we've been, and it points to where things are going. In contemplating why the line points up or down, or why it is flat, one grapples with the fundamental mechanism of what's on study.

It wasn't until I'd produced the Analytics Package graphs for several months that my manager granted me the responsibility of writing the cover letter. It was a momentous day, but the actual task of writing the summary of the state of the business wasn't hard. By looking at each graph and investigating why each had changed in which way from last month to this, I had all the key points worth writing up. Building the graphs was more than half the battle.

So many of the principles in Tufte's book made their way into the Analytics Package. For example, where relevant, each page showed a series of small multiples, with the same scale on X and Y axes, back in the age before small multiples were a thing in spreadsheet programs.

Nowhere was Tufte's influence more felt than in our line graphs. How good can a line graph be? After all, in its components, a line graph is really simple. That's a strength, not a weakness. The advice here is simple, so simple, in fact, one might think all of it is common practice already. It isn't. When I see line graphs shared online, even those from some of the smartest people I follow, almost all of them adhere to very little of what I'm going to counsel.

Perhaps Tufte isn't well read enough, his idea not taught in institutions like business schools that require their students to use Excel. That is all true, but I prefer a simpler explanation: users are lazy, the Excel line graph defaults are poor, and Excel is the most popular charting tool on the planet.

By way of illustration, let's take a data set, build a line graph in Excel, and walk through some of what I had to do when making the Analytics Package each month.

I couldn't find the raw data behind most charts shared online, and I didn't want to use any proprietary data. My friend Dan Wang pointed me at the Google Public Data Explorer, a lot of which seems to built off the World Bank Data Catalog, from which I pulled some raw data, just to save me the time of making up figures.

I used health expenditure per capita (current US$). I picked eight countries and used the data for the full range of years available, spanning 1995-2014. I chose a mix of countries I've visited or lived in, plus some people have spoken to me about in reference to their healthcare systems, but the important point here is that limiting the data series on a line graph matters if the graph is going to be readable. How many data series depends on what you want to study and how closely the lines cluster, how large the spread is. Sometimes it's hard to anticipate until you produce the graph, but suffice it to say that generating a graph just to make one is silly if the result is illegible.

Here's what the latest version of Excel on my Mac produced when I pressed the line graph button after highlighting the data (oddly enough, I found a Recommended Charts dropdown button and the three graphs it recommended were three bar graphs of a variety of forms, definitely not the right choice here, among many places where Excel's default logic is poor). I didn't do anything to this graph, just saved it directly as is, at the exact size and formatting Excel selected.

Not great. Applying Richard Thaler and Cass Sunstein's philosophy from Nudge, if we just improved the defaults in Excel and Powerpoint, the graphic excellence the world over would improve by leaps and bounds. If someone out there works on the charting features in Excel and Powerpoint, hear my cries! The power to elevate your common man is in your hands. Please read Tufte.

As an aside, after the Tufte seminar, I walked up to him and asked what software he used for the graphics in his book. His response? Adobe Illustrator. To produce the results he wanted, he, and presumably his assistants, laid out every pixel by hand. Not that helpful for me in producing the Analytics Package monthly on top of my other job duties, but a comment on the charting quality in Excel that rings true even today.

Let's take my chart above and start editing it for the better, as I did back in my Analytics Package days. Let's start with some obvious problems:

  • The legend is nearly the same height as the graph itself
  • A lot of the lines are really close to each other
  • The figures in the left column could be made more readable with thousands comma separators
  • The chart needs a title

I expanded the graph inside the worksheet to make it easier to see, it was the size of about four postage stamps in the sheet for some reason, and fixed the problems above. Here's that modified version.

Excel should add comma separators for thousands by default. The graph is somewhat better, but the labels are still really small, even if you click and expand the photo above to full size. In addition to adjusting the scale of labels and title, however, what else can we do to improve matters?

I began this post just wanting to share the following simple point, the easiest way to upgrade your Excel line graph:

Remove the legend.

That alone will make your line graph so much better that if it's the only thing you remember for the rest of your life, a generation of audiences will thank you.

The problem with a legend is that it asks the user to bounce their eyes back and forth from the graph to the legend, over and over, trying to hold what is usually some color coding system in their short-term memory.

Look at the chart above. Every time I have to see which line is which country, I have look down to the legend and then back to the graph. If I decide to compare any two data series, I have to look back down and memorize two colors, then look back at the chart. Forget even trying to do it for three countries, or for all of them, which is the whole point of the line graph. In forcing the viewer to interpret your legend, your line graph has already dampened much of its explanatory efficiency.

If you have just two data series, a legend isn't egregious, but it is still inferior to removing the legend. Of course, removing the legend isn't enough.

Remove the legend and label the data series directly on the plot.

Unfortunately, here is where your work gets harder, because, much to my disbelief, there is no option in Excel to label data series in place in a line graph. The only automated option is to use the legend.

If I am wrong, and I would love nothing more than to be wrong, please let me know, but I tried going through every one of Excel's various chart menus, which can be brought up by right clicking on different hot spots on the chart, and couldn't find this option. That Excel forces you to right click on so many obscure hot spots just to bring up various options is bad enough. That you can't find this option at all among the dozens of other useless options is a travesty.

The only fix is to create data series labels by hand. You can find an Insert Text Box option somewhere in the Excel menus and ribbons and bars, so I'll make one for each data series and put them roughly in place so you know which data series is which. Next, the moment of truth.

Select the legend. And delete it.

Undo, then delete it again, just to feel the rush as your chart now expands in size to fill the white space left behind. Feels good.

Next, shrink the actual plot area of the graph by selecting it and opening up some margin on the right side of the graph for placing your labels if there isn't enough. Since people read time series left to right, and since the most recent data is on the far right, you'll want your labels to be there, where the viewers' eyes will flow naturally.

Don't move the labels into exact position just yet. First adjust the sizing of the data labels of the axes and the scale of the graph first. Unfortunately, since these text boxes are floating and not attached to the data series, every time the scale of your chart changes, you have to reposition all the data series labels by hand. So do that last.

I have not used this latest version of Excel before, the charting options seem even more complex than before. To change the format of the labels on the x and y-axis, you right click each axis and select Format Axis. I changed the y-axis text format to currency. But to change the size of the labels on each axis, you have to right click each and then select Font. That those are in separate menus is part of the Excel experience.

In expanding the font size of the x-axis, I decided it was too crowded so I went with every other year. I left aligned the data series labels and tried to position them as precisely as possible by eye. I seem to remember Excel used to allow selecting text boxes and moving them one pixel at a time with the arrow keys, but it didn't work for me, so you may have to find the object alignment dropdown somewhere and select align left for all your labels.

Here's the next iteration of the chart.

You can click on it to see it larger. Already, we're better off than the Excel auto-generated chart by quite a margin. If this were the default, I'd be fairly happy. But there's room for improvement.

The use of color can be helpful, especially with lines that are closely stacked, but what about the color blind? If we were to stick with the coloring scheme, I might change the data series labels to match the color of each line. Again, since the labels are added by hand, you'd have to manually change each label by hand to match the color scheme Excel had selected, and again, it wouldn't fix the issue for color blind viewers. [I didn't have the patience to do this for illustrative purposes, but you can see how matching the coloring of labels to the lines helps if you view this data in Google Data Explorer.]

In The Visual Display of Quantitative Information, Tufte uses very little color. When producing the Analytics Package, I was working with black and white printers and copy machines. Color was a no go, even if it provides an added dimension for your graphics, as for elevation on maps.

While color has the advantage of making it easier to distinguish between two lines which are close to each other, it introduces all sorts of mental associations that are difficult to anticipate and which may just be a distraction. When making a chart like, say, one of the U.S. Presidential Election, using blue for Democrats and red for Republicans is a good idea since the color scheme is widely agreed upon. When distinguishing between departments in your company, or product lines, arbitrary color choices can be noise, or worse, a source of contention (NSFW language warning).

The safer alternative is to use different line styles, regardless of whether your final deliverable is capable of displaying color. Depending on how many data series you have to chart, that may or may not be an option. I looked at the data series line format options, which are labeled Dash Type in this version of Excel, and found a total of eight options, or just enough for my example chart. It takes some work to assign options for maximum legibility; you should which country receives which style based on maximum contrast between lines that cluster.

After a random pass at that, the monochrome line graph looked like this.

No issues for color blind users, but we're stretching the limits of line styles past where I'm comfortable. To me, it's somewhat easier with the colored lines above to trace different countries across time versus each other, though this monochrome version isn't terrible. Still, this chart reminds me, in many ways, of the monochromatic look of my old Amazon Analytics Package, though it is missing data labels (wouldn't fit here) and has horizontal gridlines (mine never did).

We're running into some of these tradeoffs because of the sheer number of data series in play. Eight is not just enough, it is probably too many. Past some number of data series, it's often easier and cleaner to display these as a series of small multiples. It all depends on the goal and what you're trying to communicate.

At some point, no set of principles is one size fits all, and as the communicator you have to make some subjective judgments. For example, at Amazon, I knew that Joy wanted to see the data values marked on the graph, whenever they could be displayed. She was that detail-oriented. Once I included data values, gridlines were repetitive, and y-axis labels could be reduced in number as well.

Tufte advocates reducing non-data-ink, within reason, and gridlines are often just that. In some cases, if data values aren't possible to fit onto a line graph, I sometimes include gridlines to allow for easy calculation of the relative ratio of one value to another (simply count gridlines between the values), but that's an edge case.

For sharp changes, like an anomalous reversal in the slope of a line graph, I often inserted a note directly on the graph, to anticipate and head off any viewer questions. For example, in the graph above, if fewer data series were included, but Greece remained, one might wish to explain the decline in health expenditures starting in 2008 by adding a note in the plot area near that data point, noting the beginning of the Greek financial crisis (I don't know if that's the actual cause, but whatever the reason or theory, I'd place it there).

If we had company targets for a specific metric, I'd note those on the chart(s) in question as a labeled asymptote. You can never remind people of goals often enough.

Just as an example, here's another version of that chart, with fewer data series, data labels, no gridlines, fewer y-axis labels. Also, since the lines aren't clustered together, we no longer need different line styles adding visual noise.

At that size, the data values aren't really readable, but if I were making a chart for Joy or Jeff, I'd definitely add the labels because I knew they'd want that level of detail. At Amazon, also, I typically limited our charts to rolling four or eight quarters, so we'd never have this many data points as on the graph above. Again, at some point you have to determine your audience and your goals and modify your chart to match.

Like a movie, work on a chart is a continuous process. I could generate a couple more iterations on the chart above for different purposes, but you get the idea. At some point you have to print it. Just as you'd add the end credits to a film, the last touch here would be to put a source for the data below the graph, so people can follow up on the raw data themselves.

Before I set off on this exercise, I didn't know much about health care expenditures per capita around the world, except that the United States is the world leader by a wide margin. The graph reveals that, and by what magnitude. Look at China by comparison. What explains China's low expenditures? I might hypothesize a number of reasons, including obvious ones like the huge population there, but it would take further investigation, and perhaps more charts. One reason the Analytics Package grew in time was that some charts beget further charts.

Why did Greece's expenditures per capita go into decline starting in 2008. Was it the financial crisis? Why has Japan reversed its upward trajectory starting in 2012? Should we include some other countries for comparison, and how might we choose the most illuminating set?

Every month that first year at Amazon, I'd spend most my waking hours gathering figures and confirming their accuracy, producing these graphs, and then puzzling over the stories behind their contours. The process of making line graphs was prelude to understanding.

To accelerate that understanding, upgrade your line graphs to be efficient and truthful. Some broadly applicable principles should guide you to the right neighborhood. To recap:

  • Don't include a legend; instead, label data series directly in the plot area. Usually labels to the right of the most recent data point are best. Some people argue that a legend is okay if you have more than one data series. My belief is that they're never needed on any well-constructed line graph.
  • Use thousands comma separators to make large figures easier to read
  • Related to that, never include more precision than is needed in data labels. For example, Excel often chooses two decimal places for currency formats, but most line graphs don't need that, and often you can round to 000's or millions to reduce data label size. If you're measuring figures in the billions and trillions, we don't need to see all those zeroes, in fact it makes it harder to read.
  • Format axis labels to match the format of the figures being measured; if it's US dollars, for example, format the labels as currency.
  • Look at the spacing of axis labels and increase the interval if they are too crowded. As Tufte counsels, always reduce non-data-ink as much as possible without losing communicative power.
  • Start your y-axis at zero (assuming you don't have negative values)
  • Try not to have too many data series; five to eight seems the usual limit, depending on how closely the lines cluster. On rare occasion, it's fine to exceed this; sometimes the sheer volume of data series is the point, to show a bunch of lines clustered. These are edge cases for a reason, however.
  • If you have too many data series, consider using small multiples if the situation warrants, for example if the y-axes can match in scale across all the multiples.
  • Respect color blind users and those who may not be able to see your charts with color, for example on a black and white printout, and have options for distinguishing data series beyond color, like line styles. At Amazon, as I dealt with so many figures, I always formatted negative numbers to be red and enclosed in parentheses for those who wouldn't see the figures in color.
  • Include explanations for anomalous events directly on the graph; you may not always be there in person to explain your chart if it travels to other audiences.
  • Always note, usually below the graph, the source for the data.

Some other suggestions which are sometimes applicable:

  • Display actual data values on the graph if people are just going to ask what the figures are anyway, and if they fit cleanly. If you include data labels, gridlines may not be needed. In fact, they may not be needed even if you don't include data labels.
  • Include targets for figures as asymptotes to help audiences see if you're on track to reach them.

Why is The Visual Display of Quantitative Information such a formative text in my life? If it were merely a groundbreaking book on graphic excellence, it would remain one of my trusted references, sitting next to Garner's Modern American Usage, always within arm's reach. It wouldn't be a book I would push on those who never make graphs and charts.

The reason the book influenced me so deeply is that it is actually a book about the pursuit of truth through knowledge. It is ostensibly about producing better charts; what stays with you is the principles for general clarity of thought. Reading the book, chiseling away at my line graphs late nights, talking to people all over the company to understand what might explain each of them, gave me a path towards explaining the past and predicting the future. Ask anyone about any work of art they love, whether it's a book or a movie or an album, and it's never just about what it's about. I haven't read Zen and the Art of Motorcycle Maintenance; I'm guessing it wasn't written just for motorcycle enthusiasts.

A good line graph is a fusion of right and left brain, of literacy and numeracy. Just numbers alone aren't enough to explain the truth, but accurate numbers, represented truthfully, are a check on our anecdotal excesses, confirmation biases, tribal affiliations.

I'm reminded of Tufte's book whenever I brush against tendrils of many movements experiencing a moment online: rationalism, the Nate Silver/538 school of statistics-backed journalism, infographics, UX/UI/graphic design, pop economics, big history. And, much to my dismay, I'm reminded of the book most every time I see a line graph that could use some visual editing. Most people are lazy, most people use the defaults, and the defaults of the most popular charting application on the planet, Excel, are poor.

[Some out there may ask about Apple's Numbers. I tried it a bit, and while it's aesthetically cleaner than Excel, it's such a weak spreadsheet overall that I couldn't make the switch. I dropped Powerpoint for Keynote, though both have some advantages. Neither, unfortunately, includes a great charting tool, though they are simpler in function than the one in Excel. Google Sheets is, like Numbers, a really weak spreadsheet, and it's just plain ugly. If someone out there knows of a superior charting tool, one that doesn't require making charts in Illustrator like Tufte does, please let me know.] 

I love this exchange early on in Batman Begins between Liam Neeson R'as Al Ghul (though he was undercover as Henri Ducard at the time) and Christian Bale's Bruce Wayne.

Bruce Wayne: You're vigilantes.
 
Henri Ducard: No, no, no. A vigilante is just a man lost in the scramble for his own gratification. He can be destroyed, or locked up. But if you make yourself more than just a man, if you devote yourself to an ideal, and if they can't stop you, then you become something else entirely.
 
Bruce Wayne: Which is?
 
Henri Ducard: Legend, Mr. Wayne.
 

It is absurdly self-serious in the way that nerds love their icons to be treated by mainstream pop culture, and I love it for its broad applicability. I've been known to drop some version of it in meetings all the time, my own Rickroll, but no one seems to find it amusing.

In this case, the passage needs some tweaking. But please do still read it with Liam Neeson's trademark gravitas.

A line graph is just another ugly chart lost in the scramble for its own gratification in a slide deck no one wants to read. It can be disregarded, forgotten. But if you make your graph more than just the default Excel format, if you devote yourself to Tufte's ideals, then your graph becomes something else entirely.

Which is?

A line graph without a legend. Remove the legend, Mr. Wayne, and become a legend.

Revisiting The Odyssey

What a pleasant surprise, to have something wonderful that you hadn't heard of just drop out of the sky one day, like Beyonce's Lemonade. That's how I felt about Emily Wilson's new translation of The Odyssey, which arrived on my Kindle yesterday (a physical copy isn't due on my doorstep until next week; this is the rare book for which I wanted one of each format).

So far, I've read more about Wilson's thought process behind her translation than the actual translation itself, but even that is delightful:

In planning to translate the poem into English, my first thoughts were of style. The original is written in a highly rhythmical form of verse. It reads nothing like prose and nothing like any spoken or nonpoetic kinds of discourse. Many modern poets in the Anglo-American tradition write free verse, and modern British and American readers are not usually accustomed to reading long narratives with a regular metrical beat, except for earlier literature like Shakespeare. Most contemporary translators of Homer have not attempted to create anything like a regular line beat, though they often lay out their text as if it were verse. But The Odyssey is a poem, and it needs to have a predictable and distinctive rhythm that can be easily heard when the text is read out loud. The original is in six-footed lines (dactylic hexameters), the conventional meter for archaic Greek narrative verse. I used iambic pentameter, because it is the conventional meter for regular English narrative verse—the rhythm of Chaucer, Shakespeare, Milton, Byron, Keats, and plenty of more recent anglophone poets. I have spent many hours reading aloud, both the Greek original and my own work in progress. Homer's music is quite different from mine, but my translation sings to its own regular and distinctive beat. 
 
My version is the same length as the original, with exactly the same number of lines. I chose to write within this difficult constraint because any translation without such limitations will tend to be longer than the original, and I wanted a narrative pace that could match its stride to Homer's nimble gallop. Moreover, in reading the original, one is constantly aware of the rhythms and the units that make up elements of every line, as well as of the ongoing movement of the narrative—like a large, elaborate piece of embroidery made of tiny, still visible stitches. I wanted my translation to mark its own nature as a web of poetic language, with a sentence structure that is, like that of Homer, audibly built up out of smaller units of sense. There is often a notion, especially in the Anglo-American world, that a translation is good insofar as it disguises its own existence as a translation; translations are praised for being "natural." I hope that my translation is readable and fluent, but that its literary artifice is clearly apparent. 
 
Matthew Arnold famously claimed that translators of Homer must convey four supposedly essential qualities of Homeric style: plainness, simplicity, directness of thought, and nobility. But Homeric style is actually quite often redundant and very often repetitious—not particularly simple or direct. Homer is also very often not "noble": the language is not colloquial, and it avoids obscenity, but it is not bombastic or grandiloquent. The notion that Homeric epic must be rendered in grand, ornate, rhetorically elevated English has been with us since the time of Alexander Pope. It is past time, I believe, to reject this assumption. Homer's language is markedly rhythmical, but it is not difficult or ostentatious, The Odyssey relies on coordinated, not subordinated syntax ("and then this, and then this, and then this," rather than "although this, because of that, when this, which was this, on account of that"). I have frequently aimed for a certain level of simplicity, often using fairly ordinary, straightforward, and readable English. In using language that is largely simple, my goal is not to make Homer sound "primitive," but to mark the fact that stylistic pomposity is entirely un-Homeric. I also hope to invite readers to respond more actively with the text. Impressive displays of rhetoric and linguistic force are a good way to seem important and invite a particular kind of admiration, but they tend to silence dissent and discourage deeper modes of engagement. A consistently elevated style can make it harder for readers to keep track of what is at stake in the story. My translation is, I hope, recognizable as an epic poem, but it is one that avoids trumpeting its own status with bright, noisy linguistic fireworks, in order to invite a more thoughtful consideration of what the narrative means, and the ways it matters.
 

I'm with her on all of that. Iambic pentameter! Be still my bleeding heart.

Nodding along when she makes choices like this:

The formulaic elements in Homer, especially the repeated epithets, pose a particular challenge. The epithets applied to Dawn, Athena, Hermes, Zeus, Penelope, Telemachus, Odysseus, and the suitors repeat over and over in the original. But in my version, I have chosen deliberately to interpret these epithets in several different ways, depending on the demands of the scene at hand. I do not want to deceive the unsuspecting reader about the nature of the original poem; rather, I hope to be truthful about my own text—its relationships with its readers and with the original. In an oral or semiliterate culture, repeated epithets give a listener an anchor in a quick-moving story. In a highly literate society such as our own, repetitions are likely to feel like moments to skip. They can be a mark of writerly laziness or unwillingness to acknowledge one's own interpretative position, and can send a reader to sleep. I have used the opportunity offered by the repetitions to explore the multiple different connotations of each epithet.
 

I try not to spend too much time fetishizing craft, but when it comes to translation, it is inseparable from the thing.

This past Sunday's NYTimes Magazine included a feature on Wilson's accomplishment. In this age where we celebrate women breaking through in fields previously occupied by only white men, being the first woman to translate one of the great works of Western literature resounds at many levels.

The NYTimes feature spends some time laying bare the translator's hand. Take, for example, the thought that went into the opening line of the epic itself, and how varied its forms in all the translations that had been come before Wilson's. Just one line and already so many forks.

The first of these changes is in the very first line. You might be inclined to suppose that, over the course of nearly half a millennium, we must have reached a consensus on the English equivalent for an old Greek word, polytropos. But to consult Wilson’s 60 some predecessors, living and dead, is to find that consensus has been hard to come by. Chapman starts things off, in his version, with “many a way/Wound with his wisdom”; John Ogilby counters with the terser “prudent”; Thomas Hobbes evades the word, just calling Odysseus “the man.” Quite a range, and we’ve barely started. There’s Alexander Pope’s “for wisdom’s various arts renown’d”; William Cowper’s “For shrewdness famed/And genius versatile”; H.F. Cary’s “crafty”; William Sotheby’s “by long experience tried”; Theodore Buckley’s “full of resources”; Henry Alford’s “much-versed”; Philip Worsley’s “that hero”; the Rev. John Giles’s “of many fortunes”; T.S. Norgate’s “of many a turn”; George Musgrave’s “tost to and fro by fate”; the Rev. Lovelace Bigge-Wither’s “many-sided-man”; George Edgington’s “deep”; William Cullen Bryant’s “sagacious”; Roscoe Mongan’s “skilled in expedients”; Samuel Henry Butcher and Andrew Lang’s “so ready at need”; Arthur Way’s “of craft-renown”; George Palmer’s “adventurous”; William Morris’s “shifty”; Samuel Butler’s “ingenious”; Henry Cotterill’s “so wary and wise”; Augustus Murray’s “of many devices”; Francis Caulfeild’s “restless”; Robert Hiller’s “clever”; Herbert Bates’s “of many changes”; T.E. Lawrence’s “various-minded”; William Henry Denham Rouse’s “never at a loss”; Richmond Lattimore’s “of many ways”; Robert Fitzgerald’s “skilled in all ways of contending”; Albert Cook’s “of many turns”; Walter Shewring’s “of wide-ranging spirit”; Allen Mandelbaum’s “of many wiles”; Robert Fagles’s “of twists and turns”; all the way to Stanley Lombardo’s “cunning.”
 
Of the 60 or so answers to the polytropos question to date, the 36 given above couldn’t be less uniform (the two dozen I omit repeat, with minor variations, earlier solutions); what unites them is that their translators largely ignore the ambiguity built into the word they’re translating. Most opt for straightforward assertions of Odysseus’s nature, descriptions running from the positive (crafty, sagacious, versatile) to the negative (shifty, restless, cunning). Only Norgate (“of many a turn”) and Cook (“of many turns”) preserve the Greek roots as Wilson describes them — poly (“many”), tropos (“turn”) — answers that, if you produced them as a student of classics, much of whose education is spent translating Greek and Latin and being marked correct or incorrect based on your knowledge of the dictionary definitions, would earn you an A. But to the modern English reader who does not know Greek, does “a man of many turns” suggest the doubleness of the original word — a man who is either supremely in control of his life or who has lost control of it? Of the existing translations, it seems to me that none get across to a reader without Greek the open question that, in fact, is the opening question of the “Odyssey,” one embedded in the fifth word in its first line: What sort of man is Odysseus?
 

All that variation in just one word. Let's telescope out to the entire opening paragraph or stanza (the Paris Review published an excerpt from the opening of Wilson's translation), the invocation of the Muse:

Tell me about a complicated man.
Muse, tell me how he wandered and was lost
when he had wrecked the holy town of Troy,
and where he went, and who he met, the pain
he suffered in the storms at sea, and how
he worked to save his life and bring his men
back home. He failed to keep them safe; poor fools,
they ate the Sun God’s cattle, and the god
kept them from home. Now goddess, child of Zeus,
tell the old story for our modern times.
Find the beginning.
 

Here is how another popular translation, by Robert Fagles, handles the same passage:

Sing to me of the man, Muse, the man of twists and turns ...
driven time and again off course, once he had plundered
the hallowed heights of Troy.
Many cities of men he saw and learned their minds,
many pains he suffered, heartsick on the open sea,
fighting to save his life and bring his comrades home.
But he could not save them from disaster, hard as he strove—
the recklessness of their own ways destroyed them all,
the blind fools, they devoured the cattle of the Sun
and the Sungod blotted out the day of their return.
Launch out on his story, Muse, daughter of Zeus,
start from where you will—sing for our time too.
 

Here is Richmond Lattimore's opening. I can't remember if I had to read this or Fagles' in high school or college, it was one or the other.

Tell me, Muse, of the man of many ways, who was driven
far journeys, after he had sacked Troy’s sacred citadel.
Many were they whose cities he saw, whose minds he learned of,
many the pains he suffered in his spirit on the wide sea,
struggling for his own life and the homecoming of his companions.
Even so he could not save his companions, hard though
he strove to; they were destroyed by their own wild recklessness,
fools, who devoured the oxen of Helios, the Sun God,
and he took away the day of their homecoming.
From some point here, goddess, daughter of Zeus,
speak, and begin our story.
 

While they all have their virtues, it's impossible to ignore the startling directness of Wilson's version. It is direct, more concise, and has a lyrical momentum from the iambic pentameter that adds to its muscularity. "He failed to keep them safe" is stronger in tone than "he could not save them from disaster, hard as he strove" or "he could not save his companions, hard though he strove to." Wilson may choose to leave out the striving because, in the previous sentence, as all the translations include, it is already noted that Odysseus suffered many pains to save his life and bring his men home. How hard he strove is somewhat repetitive, so Wilson just nixes it.

"Complicated man" is about as tidy a way to characterize a character who contains multitudes. Fagles' "the man of twists and turns" doesn't register much to me except the image of a pretzel. Lattimore's "the man of many ways" is intriguing, less precise than Wilson's "complicated man" but hinting at both Odysseus' resourcefulness and complexity.

Direct does not mean Wilson dispenses with fun. Later in the opening we have this:

So why do you dismiss Odysseus?”
 

Let your tongue tap over that like a rock skip-skipping o'er a pond.

I could not help flipping ahead to what I consider one of the most iconic scenes in Western literature, in which Odysseus and his men, having blinded the one-eyed monster Polyphemus, are in a boat, escaping the island where the creature had held them hostage. Polyphemus does not know where they are, he is beside himself with pain and fury. Just as Odysseus and his men are about to escape unharmed, he turns back to face his vanquished foe. He cannot help himself.

When I had gone as far as shouts can carry,
I jeered back,

‘Hey, you, Cyclops! Idiot!
The crew trapped in your cave did not belong
to some poor weakling. Well, you had it coming!
You had no shame at eating your own guests!
So Zeus and other gods have paid you back.’

My taunting made him angrier. He ripped
a rock out of the hill and hurled it at us.
It landed right in front of our dark prow,
and almost crushed the tip of the steering oar.
The stone sank in the water; waves surged up.
The backflow all at once propelled the ship
landwards; the swollen water pushed us with it.
I grabbed a big long pole, and shoved us off.
I told my men, ‘Row fast, to save your lives!’
and gestured with my head to make them hurry.
They bent down to their oars and started rowing.
We got out twice as far across the sea,
and then I called to him again. My crew
begged me to stop, and pleaded with me.

‘Please!
Calm down! Why are you being so insistent
and taunting this wild man? He hurled that stone
and drove our ship right back to land. We thought
that we were going to die. If he had heard us,
he would have hurled a jagged rock and crushed
our heads and wooden ship. He throws so hard!

But my tough heart was not convinced; I was
still furious, and shouted back again,

‘Cyclops! If any mortal asks you how
your eye was mutilated and made blind,
say that Odysseus, the city-sacker,
Laertes’ son, who lives in Ithaca,
destroyed your sight.’ 
 

This is, for my money, one of the seminal moments in Western literature. It's the birth of ritualized trash talk and boasting, the defining instance of taunting in the Western canon. Every time you see a basketball player get all up in the mug of some opponent after dunking on them, every time Cam Newton pantomines opening his jersey to reveal the Superman cape, every time a rapper refers to himself in the third person after performing lyrical violence on a nemesis, every time Roy Jones Jr. gave a shout out to Pensacola after one of his boxing victories, it all traces back to this moment when Odysseus can't help claiming personal credit for having outwitted the beast, giving himself a title (city-sacker) and naming himself in relation to his family (Laertes' son) and his home (Ithaca).

When Danaerys Targaryen on Game of Thrones introduces herself as "Daenerys Stormborn of the House Targaryen, First of Her Name, the Unburnt, Queen of the Andals and the First Men, Khaleesi of the Great Grass Sea, Breaker of Chains, and Mother of Dragons," she should credit Odysseus,, the city-sacker, Laertes' son, who lives in Ithaca. When we use the term "making our name," we call back to Odysseus, who in that moment established the now familiar tradition of referring to oneself in the third person.

It comes with a cost. His pride and arrogance not only endanger his men by revealing their location and allowing Polyphemus to better target his next rock throw, but in making his name, Odysseus gives Polyphemus a specific target. As anyone who has read mythology or fairy tales knows, a specific name is needed to target curses from afar. Polyphemus' father just happens to be Poseidon, a god, and it's to poppa that he turns for help.

But he prayed
holding his arms towards the starry sky,
‘Listen, Earth-Shaker, Blue-Haired Lord Poseidon:
acknowledge me your son, and be my father.
Grant that Odysseus, the city-sacker,
will never go back home. Or if it is
fated that he will see his family,
then let him get there late and with no honor,
in pain and lacking ships, and having caused
the death of all his men, and let him find
more trouble in his own house.’

Blue Poseidon
granted his son’s prayer.
 

And so Odysseus brings a curse upon himself, his family, and his men. All of the above comes true, as prophecies are wont to do in stories like this.

I'd be inclined to chide him for his hubris, but wouldn't the internet be better today if trolls didn't hide like cowards behind the veil of anonymity? Face your critics, and name yourself, anonymous neo-Nazis. Rereading this passage, I couldn't help but think of Peter Cvjetanovic, the student who marched in the Charlottesville protests and was identified in a photo.

I am Peter Cvjetanovic,
Charlottesville city sacker,
neo-Nazi sympathizer
University Nevada Reno student
and campus escort service driver,
or used to be.

Let's not end there. Let's leave with a more pleasant example, when Russell Crowe removes his mask in perhaps the most iconic arena of battle in Western myth, the Colosseum, and names himself, as his ancestor Odysseus once did. If you're looking for inspiration for the next draft of your Twitter bio, now you have it, all you tweeter of tweets.

10 browser tabs

1. Love in the Time of Robots

“Is it difficult to play with her?” the father asks. His daughter looks to him, then back at the android. Its mouth begins to open and close slightly, like a dying fish. He laughs. “Is she eating something?”
 
The girl does not respond. She is patient and obedient and listens closely. But something inside is telling her to resist. 
 
“Do you feel strange?” her father asks. Even he must admit that the robot is not entirely believable.
 
Eventually, after a few long minutes, the girl’s breathing grows heavier, and she announces, “I am so tired.” Then she bursts into tears.
 
That night, in a house in the suburbs, her father uploads the footage to his laptop for posterity. His name is Hiroshi Ishi­guro, and he believes this is the first record of a modern-day android.
 

Reads like the treatment for a science fiction film, some mashup of Frankenstein, Pygmalion, and Narcissus. One incredible moment after another, and I'll grab just a few excerpts, but the whole thing is worth reading.

But he now wants something more. Twice he has witnessed others have the opportunity, however confusing, to encounter their robot self, and he covets that experience. Besides, his daughter was too young, and the newscaster, though an adult, was, in his words, merely an “ordinary” person: Neither was able to analyze their android encounter like a trained scientist. A true researcher should have his own double. Flashing back to his previous life as a painter, Ishi­guro thinks: This will be another form of self-portrait. He gives the project his initials: Geminoid HI. His mechanical twin.
 

Warren Ellis, in a recent commencement speech delivered at the University of Essex, said:

Nobody predicted how weird it’s gotten out here.  And I’m a science fiction writer telling you that.  And the other science fiction writers feel the same.  I know some people who specialized in near-future science fiction who’ve just thrown their hands up and gone off to write stories about dragons because nobody can keep up with how quickly everything’s going insane.  It’s always going to feel like being thrown in the deep end, but it’s not always this deep, and I’m sorry for that.
 

The thing is, far future sci-fi is likely to be even more off base now given how humans are evolving in lock step with the technology around them. So we need more near future sci-fi, of a variety smarter than Black Mirror, to grapple with the implications.

Soon his students begin comparing him to the Geminoid—“Oh, professor, you are getting old,” they tease—and Ishi­guro finds little humor in it. A few years later, at 46, he has another cast of his face made, to reflect his aging, producing a second version of HI. But to repeat this process every few years would be costly and hard on his vanity. Instead, Ishi­guro embraces the logi­cal alternative: to alter his human form to match that of his copy. He opts for a range of cosmetic procedures—laser treatments and the injection of his own blood cells into his face. He also begins watching his diet and lifting weights; he loses about 20 pounds. “I decided not to get old anymore,” says Ishi­guro, whose English is excellent but syntactically imperfect. “Always I am getting younger.”
 
Remaining twinned with his creation has become a compulsion. “Android has my identity,” he says. “I need to be identical with my android, otherwise I’m going to lose my identity.” I think back to another photo of his first double’s construction: Its robot skull, exposed, is a sickly yellow plastic shell with openings for glassy teeth and eyeballs. When I ask what he was thinking as he watched this replica of his own head being assembled, Ishi­guro says, perhaps only half-joking, “I thought I might have this kind of skull if I removed my face.”
 
Now he points at me. “Why are you coming here? Because I have created my copy. The work is important; android is important. But you are not interested in myself.”
 

This should be some science fiction film, only I'm not sure who our great science fiction director is. The best examples may be too old to want to look upon such a story as anything other than grotesque and horrific.

2. Something is wrong on the internet by James Bridle

Of course, some of what's on the internet really is grotesque and horrific. 

Someone or something or some combination of people and things is using YouTube to systematically frighten, traumatise, and abuse children, automatically and at scale, and it forces me to question my own beliefs about the internet, at every level. 
 

Given how much my nieces love watching product unwrapping and Peppa the Pig videos on YouTube, this story was induced a sense of dread I haven't felt since the last good horror film I watched, which I can't remember anymore since the world has run a DDOS on my emotions.

We often think of a market operating at peak efficiency as sending information back and forth between supply and demand, allowing the creation of goods that satisfy both parties. In the tech industry, the wink-wink version of that is saying that pornography leads the market for any new technology, solving, as it does, the two problems the internet is said to solve better, at scale, than any medium before it: loneliness and boredom.

Bridle's piece, however, finds the dark cul-de-sacs and infected runaway processes which have branched out from the massive marketplace that is YouTube. I decided to follow a Peppa the Pig video on the service and started tapping on Related Videos, like I imagine one of my nieces doing, and quickly wandered into a dark alleyway where I saw some video which I would not want any of them watching. As Bridle did, I won't link to what I found; suffice to say it won't take you long to stumble on some of it if you want, or perhaps even if you don't.

What's particularly disturbing is the somewhat bizarre, inexplicably grotesque nature of some of these video remixes. David Cronenberg is known for his body horror films; these YouTube videos are like some perverse variant of that, playing with popular children's iconography.

Facebook and now Twitter are taking heat for disseminating fake news, and that is certainly a problem worth debating, but with that problem we're talking about adults. Children don't have the capacity to comprehend what they're seeing, and given my belief in the greater effect of sight, sound, and motion, I am even more disturbed by this phenomenon.

A system where it's free to host videos to a global audience, where this type of trademark infringement weaponizes brand signifiers with seeming impunity, married with increasingly scalable content production and remixes using technology, allows for the type of scalable problem we haven't seen before.

The internet has enabled all types of wonderful things at scale; we should not be surprised that it would foster the opposite. But we can, and should, be shocked.

3. FDA approves first blood sugar monitor without finger pricks

This is exciting. One view which seems to be common wisdom these days when it comes to health is that it's easier to lose weight and impact your health through diet than exercise. But one of the problems of the feedback loop in diet (and exercise, actually) is how slow it is. You sneak a few snacks here and there walking by the company cafeteria every day, and a month later you hop on the scale and emit a bloodcurdling scream as you realize you've gained 8 pounds.

A friend of mine had gestational diabetes during one of her pregnancies and got a home blood glucose monitor. You had to prick your finger and draw blood to get your blood glucose reading, but curious, I tried it before and after a BBQ.

To see what various foods did to my blood sugar in near real-time was a real eye-opener. Imagine in the future when one could see what a few french fries and gummy bears did to your blood sugar, or when the reading could be built into something like an Apple Watch, without having to draw blood each time. I don't mind the sight of blood, but I'd prefer not to turn my finger tips into war zones.

Faster feedback might transform dieting into something more akin to deliberate practice. Given that another popular theory of obesity is that it's an insulin phenomenon, tools like this, built for diabetes, might have much mass market impact.

4.  Ingestable ketones

Ingestable ketones have been a recent sort of holy grail for endurance athletes, and now HVMN is bringing one to market. Ketogenic diets are all the rage right now, but for an endurance athlete, the process of being able to fuel oneself on ketones has always sounded like a long and miserable process.

The body generates ketones from fat when low on carbs or from fasting. The theory is that endurance athletes using ketones rather than glycogen from carbs require less oxygen and thus can work out longer.

I first heard about the possibility of exogenous ketones for athletes from Peter Attia. As he said then, perhaps the hardest thing about ingesting exogenous ketones is the horrible taste, which caused him to gag and nearly vomit in his kitchen. It doesn't sound like the taste problem has been solved.

Until we get the pill that renders exercise obsolete, however, I'm curious to give this a try. If you decide to pre-order, you can use my referral code to get $15 off.

5. We Are Nowhere Close to the Limits of Athletic Performance

By comparison, the potential improvements achievable by doping effort are relatively modest. In weightlifting, for example, Mike Israetel, a professor of exercise science at Temple University, has estimated that doping increases weightlifting scores by about 5 to 10 percent. Compare that to the progression in world record bench press weights: 361 pounds in 1898, 363 pounds in 1916, 500 pounds in 1953, 600 pounds in 1967, 667 pounds in 1984, and 730 pounds in 2015. Doping is enough to win any given competition, but it does not stand up against the long-term trend of improving performance that is driven, in part, by genetic outliers. As the population base of weightlifting competitors has increased, outliers further and further out on the tail of the distribution have appeared, driving up world records.
 
Similarly, Lance Armstrong’s drug-fuelled victory of the 1999 Tour de France gave him a margin of victory over second-place finisher Alex Zulle of 7 minutes, 37 seconds, or about 0.1 percent.3 That pales in comparison to the dramatic secular increase in speeds the Tour has seen over the past half century: Eddy Merckx won the 1971 tour, which was about the same distance as the 1999 tour, in a time 5 percent worse than Zulle’s. Certainly, some of this improvement is due to training methods and better equipment. But much of it is simply due to the sport’s ability to find competitors of ever more exceptional natural ability, further and further out along the tail of what’s possible.
 

In the Olympics, to take the most celebrated athletic competition, victors are celebrated with videos showing them swimming laps, tossing logs in a Siberian tundra, running through a Kenyan desert. We celebrate the work, the training. Good genes are given narrative short shrift. Perhaps we should show a picture of their DNA, just to give credit where much credit is due?

If I live a normal human lifespan, I expect to live to see special sports leagues and divisions created for athletes who've undergone genetic modification in the future. It will be the return of the freak show at the circus, but this time for real. I've sat courtside and seen people like Lebron James, Giannis Antetokounmpo, Kevin Durant, and Joel Embiid walk by me. They are freaks, but genetic engineering might produce someone who stretch our definition of outlier.

In other words, it is highly unlikely that we have come anywhere close to maximum performance among all the 100 billion humans who have ever lived. (A completely random search process might require the production of something like a googol different individuals!)
 
But we should be able to accelerate this search greatly through engineering. After all, the agricultural breeding of animals like chickens and cows, which is a kind of directed selection, has easily produced animals that would have been one in a billion among the wild population. Selective breeding of corn plants for oil content of kernels has moved the population by 30 standard deviations in roughly just 100 generations.6 That feat is comparable to finding a maximal human type for a specific athletic event. But direct editing techniques like CRISPR could get us there even faster, producing Bolts beyond Bolt and Shaqs beyond Shaq.
 

6. Let's set half a percent as the standard for statistical significance

My many-times-over coauthor Dan Benjamin is the lead author on a very interesting short paper "Redefine Statistical Significance." He gathered luminaries from many disciplines to jointly advocate a tightening of the standards for using the words "statistically significant" to results that have less than a half a percent probability of occurring by chance when nothing is really there, rather than all results that—on their face—have less than a 5% probability of occurring by chance. Results with more than a 1/2% probability of occurring by chance could only be called "statistically suggestive" at most. 
 
In my view, this is a marvelous idea. It could (a) help enormously and (b) can really happen. It can really happen because it is at heart a linguistic rule. Even if rigorously enforced, it just means that editors would force people in papers to say "statistically suggestive for a p of a little less than .05, and only allow the phrase "statistically significant" in a paper if the p value is .005 or less. As a well-defined policy, it is nothing more than that. Everything else is general equilibrium effects.
 

Given the replication crisis has me doubting almost every piece of conventional wisdom I've inherited in my life, I'm okay with this.

7. We're surprisingly unaware of when our own beliefs change

If you read an article about a controversial issue, do you think you’d realise if it had changed your beliefs? No one knows your own mind like you do – it seems obvious that you would know if your beliefs had shifted. And yet a new paper in The Quarterly Journal of Experimental Psychology suggests that we actually have very poor “metacognitive awareness” of our own belief change, meaning that we will tend to underestimate how much we’ve been swayed by a convincing article.
 
The researchers Michael Wolfe and Todd Williams at Grand Valley State University said their findings could have implications for the public communication of science. “People may be less willing to meaningfully consider belief inconsistent material if they feel that their beliefs are unlikely to change as a consequence,” they wrote.
 

Beyond being an interesting result, I link to this as an example of a human readable summary of a research paper. This his how this article summarize the research study and its results:

The researchers recruited over two hundred undergrads across two studies and focused on their beliefs about whether the spanking/smacking of kids is an effective form of discipline. The researchers chose this topic deliberately in the hope the students would be mostly unaware of the relevant research literature, and that they would express a varied range of relatively uncommitted initial beliefs.
 
The students reported their initial beliefs about whether spanking is an effective way to discipline a child on a scale from “1” completely disbelieve to “9” completely believe. Several weeks later they were given one of two research-based texts to read: each was several pages long and either presented the arguments and data in favour of spanking or against spanking. After this, the students answered some questions to test their comprehension and memory of the text (these measures varied across the two studies). Then the students again scored their belief in whether spanking is effective or not (using the same 9-point scale as before). Finally, the researchers asked them to recall what their belief had been at the start of the study.
 
The students’ belief about spanking changed when they read a text that argued against their own initial position. Crucially, their memory of their initial belief was shifted in the direction of their new belief – in fact, their memory was closer to their current belief than their original belief. The more their belief had changed, the larger this memory bias tended to be, suggesting the students were relying on their current belief to deduce their initial belief. The memory bias was unrelated to the measures of how well they’d understood or recalled the text, suggesting these factors didn’t play a role in memory of initial belief or awareness of belief change.
 

Compare this link above to the abstract of the paper itself:

When people change beliefs as a result of reading a text, are they aware of these changes? This question was examined for beliefs about spanking as an effective means of discipline. In two experiments, subjects reported beliefs about spanking effectiveness during a prescreening session. In a subsequent experimental session, subjects read a one-sided text that advocated a belief consistent or inconsistent position on the topic. After reading, subjects reported their current beliefs and attempted to recollect their initial beliefs. Subjects reading a belief inconsistent text were more likely to change their beliefs than those who read a belief consistent text. Recollections of initial beliefs tended to be biased in the direction of subjects’ current beliefs. In addition, the relationship between the belief consistency of the text read and accuracy of belief recollections was mediated by belief change. This belief memory bias was independent of on-line text processing and comprehension measures, and indicates poor metacognitive awareness of belief change.
 

That's actually one of the better research abstracts you'll read and still it reflects the general opacity of the average research abstract. I'd argue that some of the most important knowledge in the world is locked behind abstruse abstracts.

Why do researchers write this way? Most tell me that researchers write for other researchers, and incomprehensible prose like this impresses their peers. What a tragedy. As my longtime readers know, I'm a firm believer in the power of the form of a message. We continue to underrate that in all aspects of life, from the corporate world to our personal lives, and here, in academia.

Then again, such poor writing keeps people like Malcolm Gladwell busy transforming such insight into breezy reads in The New Yorker and his bestselling books.

8. Social disappointment explains chimpanzees' behaviour in the inequity aversion task

As an example of the above phenomenon, this paper contains an interesting conclusion, but try to parse this abstract:

Chimpanzees’ refusal of less-preferred food when an experimenter has previously provided preferred food to a conspecific has been taken as evidence for a sense of fairness. Here, we present a novel hypothesis—the social disappointment hypothesis—according to which food refusals express chimpanzees' disappointment in the human experimenter for not rewarding them as well as they could have. We tested this hypothesis using a two-by-two design in which food was either distributed by an experimenter or a machine and with a partner present or absent. We found that chimpanzees were more likely to reject food when it was distributed by an experimenter rather than by a machine and that they were not more likely to do so when a partner was present. These results suggest that chimpanzees’ refusal of less-preferred food stems from social disappointment in the experimenter and not from a sense of fairness.
 

Your average grade school English teacher would slap a failing grade on this butchery of the English language.

9. Metacompetition: Competing Over the Game to be Played

When CDMA-based technologies took off in the US, companies like QualComm that work on that standard prospered; metacompetitions between standards decide the fates of the firms that adopt (or reject) those standards.

When an oil spill raises concerns about the environment, consumers favor businesses with good environmental records; metacompetitions between beliefs determine the criteria we use to evaluate whether a firm is “good.”

If a particular organic foods certification becomes important to consumers, companies with that certification are favored; metacompetitions between certifications determines how the quality of firms is measured.
 
In all these examples, you could be the very best at what you do, but lose in the metacompetition over what criteria will matter. On the other hand, you may win due to a metacompetition that protects you from fierce rivals who play a different game.
 
Great leaders pay attention to metacompetition. They advocate the game they play well, promoting criteria on which they measure up. By contrast, many failed leaders work hard at being the best at what they do, only to throw up their hands in dismay when they are not even allowed to compete. These losers cannot understand why they lost, but they have neglected a fundamental responsibility of leadership. It is not enough to play your game well. In every market in every country, alternative “logics” vie for prominence. Before you can win in competition, you must first win the metacompetition over the game being played.
 

In sports negotiations between owners and players, the owners almost always win the metacompetition game. In the writer's strike in Hollywood in 2007, the writer's guild didn't realize they were losing the metacompetition and thus ended up worse off than before. Amazon surpassed eBay by winning the retail metacompetition (most consumers prefer paying a good, fixed price for a good of some predefined quality than dealing with the multiple axes of complexity of an auction) after first failing at tackling eBay on its direct turf of auctions.

Winning the metacompetition means first being aware of what it is. It's not so easy in a space like, say, social networking, where even some of the winners don't understand what game they're playing.

10. How to be a Stoic

Much of Epictetus’ advice is about not getting angry at slaves. At first, I thought I could skip those parts. But I soon realized that I had the same self-recriminatory and illogical thoughts in my interactions with small-business owners and service professionals. When a cabdriver lied about a route, or a shopkeeper shortchanged me, I felt that it was my fault, for speaking Turkish with an accent, or for being part of an élite. And, if I pretended not to notice these slights, wasn’t I proving that I really was a disengaged, privileged oppressor? Epictetus shook me from these thoughts with this simple exercise: “Starting with things of little value—a bit of spilled oil, a little stolen wine—repeat to yourself: ‘For such a small price, I buy tranquillity.’ ”
 
Born nearly two thousand years before Darwin and Freud, Epictetus seems to have anticipated a way out of their prisons. The sense of doom and delight that is programmed into the human body? It can be overridden by the mind. The eternal war between subconscious desires and the demands of civilization? It can be won. In the nineteen-fifties, the American psychotherapist Albert Ellis came up with an early form of cognitive-behavioral therapy, based largely on Epictetus’ claim that “it is not events that disturb people, it is their judgments concerning them.” If you practice Stoic philosophy long enough, Epictetus says, you stop being mistaken about what’s good even in your dreams.
 

The trendiness of stoicism has been around for quite some time now. I found this tab left over from 2016, and I'm sure Tim Ferriss was espousing it long before then, and not to mention the enduring trend that is Buddhism. That meditation and stoicism are so popular in Silicon Valley may be a measure of the complacency of the region; these seem direct antidotes to the most first world of problems. People everywhere complain of the stresses on their mind from the deluge of information they receive for free from apps on the smartphone with processing power that would put previous supercomputers to shame.

Still, given that stoicism was in vogue in Roman times, it seems to have stood the test of time. Since social media seems to have increased the surface area of our social fabric and our exposure to said fabric, perhaps we could all use a bit more stoicism in our lives. I suspect one reason Curb Your Enthusiasm curdles in the mouth more than before is not just that his rich white man's complaints seem particularly ill timed in the current environment but that he is out of touch with the real nature of most people's psychological stressors now. A guy of his age and wealth probably doesn't spend much time on social media, but if he did, he might realize his grievances no longer match those of the average person in either pettiness or peculiarity.

Chasm of comprehension

Last year, Google's AI AlphaGo beat Korean Lee Sedol in Go, a game many expected humans to continue to dominate for years, if not decades, to come.

With the 37th move in the match’s second game, AlphaGo landed a surprise on the right-hand side of the 19-by-19 board that flummoxed even the world’s best Go players, including Lee Sedol. “That’s a very strange move,” said one commentator, himself a nine dan Go player, the highest rank there is. “I thought it was a mistake,” said the other. Lee Sedol, after leaving the match room, took nearly fifteen minutes to formulate a response. Fan Gui—the three-time European Go champion who played AlphaGo during a closed-door match in October, losing five games to none—reacted with incredulity. But then, drawing on his experience with AlphaGo—he has played the machine time and again in the five months since October—Fan Hui saw the beauty in this rather unusual move
 
Indeed, the move turned the course of the game. AlphaGo went on to win Game Two, and at the post-game press conference, Lee Sedol was in shock. “Yesterday, I was surprised,” he said through an interpreter, referring to his loss in Game One. “But today I am speechless. If you look at the way the game was played, I admit, it was a very clear loss on my part. From the very beginning of the game, there was not a moment in time when I felt that I was leading.”
 

The first time Gary Kasparov sensed deep intelligence in Deep Blue, he described the computer's move as a very human one

I GOT MY FIRST GLIMPSE OF ARTIFICIAL INTELLIGENCE ON Feb. 10, 1996, at 4:45 p.m. EST, when in the first game of my match with Deep Blue, the computer nudged a pawn forward to a square where it could easily be captured. It was a wonderful and extremely human move. If I had been playing White, I might have offered this pawn sacrifice. It fractured Black's pawn structure and opened up the board. Although there did not appear to be a forced line of play that would allow recovery of the pawn, my instincts told me that with so many "loose" Black pawns and a somewhat exposed Black king, White could probably recover the material, with a better overall position to boot.
 
But a computer, I thought, would never make such a move. A computer can't "see" the long-term consequences of structural changes in the position or understand how changes in pawn formations may be good or bad.
 
Humans do this sort of thing all the time. But computers generally calculate each line of play so far as possible within the time allotted. Because chess is a game of virtually limitless possibilities, even a beast like Deep Blue, which can look at more than 100 million positions a second, can go only so deep. When computers reach that point, they evaluate the various resulting positions and select the move leading to the best one. And because computers' primary way of evaluating chess positions is by measuring material superiority, they are notoriously materialistic. If they "understood" the game, they might act differently, but they don't understand.
 
So I was stunned by this pawn sacrifice. What could it mean? I had played a lot of computers but had never experienced anything like this. I could feel--I could smell--a new kind of intelligence across the table. While I played through the rest of the game as best I could, I was lost; it played beautiful, flawless chess the rest of the way and won easily.
 

Later, in the Kasparov-Deep Blue rematch that IBM's computer won, again a move in the 2nd game was pivotal. There is debate or whether the move was a mistake or intentional on the part of the computer, but it flummoxed Kasparov (italics mine):

'I was not in the mood of playing at all,'' he said, adding that after Game 5 on Saturday, he had become so dispirited that he felt the match was already over. Asked why, he said: ''I'm a human being. When I see something that is well beyond my understanding, I'm afraid.''
 
...
 

At the news conference after the game, a dark-eyed and brooding champion said that his problems began after the second game, won by Deep Blue after Mr. Kasparov had resigned what was eventually shown to be a drawn position. Mr. Kasparov said he had missed the draw because the computer had played so brilliantly that he thought it would have obviated the possibility of the draw known as perpetual check.

''I do not understand how the most powerful chess machine in the world could not see simple perpetual check,'' he said. He added he was frustrated by I.B.M.'s resistance to allowing him to see the printouts of the computer's thought processes so he could understand how it made its decisions, and implied again that there was some untoward behavior by the Deep Blue team.

Asked if he was accusing I.B.M. of cheating, he said: ''I have no idea what's happening behind the curtain. Maybe it was an outstanding accomplishment by the computer. But I don't think this machine is unbeatable.''

Mr. Kasparov, who defeated a predecessor of Deep Blue a year ago, won the first game of this year's match, but it was his last triumph, a signal that the computer's pattern of thought had eluded him. He couldn't figure out what its weaknesses were, or if he did, how to exploit them.

Legend has it that a move in Game One and another in Game Two were actually just programming glitches that caused Deep Blue to make random moves that threw Kasparov off, but regardless, the theme is the same: at some point he no longer understood what the program was doing. He no longer had a working mental model, like material advantage, for his computer opponent.

This year, a new version of AlphaGo was unleashed on the world: AlphaGo Zero.

As many will remember, AlphaGo—a program that used machine learning to master Go—decimated world champion Ke Jie earlier this year. Then, the program’s creators at Google’s DeepMind let the program continue to train by playing millions of games against itself. In a paper published in Nature earlier this week, DeepMind revealed that a new version of AlphaGo (which they christened AlphaGo Zero) picked up Go from scratch, without studying any human games at all. AlphaGo Zero took a mere three days to reach the point where it was pitted against an older version of itself and won 100 games to zero.
 

(source)

That AlphaGo Zero had nothing to learn from playing the world's best humans, and that it trounced its artificial parent 100-0, is evolutionary velocity of a majesty not seen since the ectomorphs in the Alien movie franchise. It is also, in its arrogance, terrifying.

DeepMind released 55 games that a previous version of AlphaGo played against itself for Go players around the world to analyze.

Since May, experts have been painstakingly analyzing the 55 machine-versus-machine games. And their descriptions of AlphaGo’s moves often seem to keep circling back to the same several words: Amazing. Strange. Alien.
 
“They’re how I imagine games from far in the future,” Shi Yue, a top Go player from China, has told the press. A Go enthusiast named Jonathan Hop who’s been reviewing the games on YouTube calls the AlphaGo-versus-AlphaGo face-offs “Go from an alternate dimension.” From all accounts, one gets the sense that an alien civilization has dropped a cryptic guidebook in our midst: a manual that’s brilliant—or at least, the parts of it we can understand.
 
[...]
 

Some moves AlphaGo likes to make against its clone are downright incomprehensible, even to the world’s best players. (These tend to happen early on in the games—probably because that phase is already mysterious, being farthest away from any final game outcome.) One opening move in Game One has many players stumped. Says Redmond, “I think a natural reaction (and the reaction I’m mostly seeing) is that they just sort of give up, and sort of throw their hands up in the opening. Because it’s so hard to try to attach a story about what AlphaGo is doing. You have to be ready to deny a lot of the things that we’ve believed and that have worked for us.”

 

Like others, Redmond notes that the games somehow feel “alien.” “There’s some inhuman element in the way AlphaGo plays,” he says, “which makes it very difficult for us to just even sort of get into the game.”
 

Ke Jie, the Chinese Go master who was defeated by AlphaGo earlier this year, said:

Last year, it was still quite humanlike when it played. But this year, it became like a god of Go.”
 

After his defeat, Ke posted what might be the most poetic and bracing quote of 2017 on Weibo (I first saw it in the WSJ):

“I would go as far as to say not a single human has touched the edge of the truth of Go.”
 

***

When Josh Brown died in his Tesla after driving under a semi, it kicked off a months long investigation into who was at fault. Ultimately, the NHTSA absolved Autopilot of blame. The driver was said to have had 7 seconds to see the semi and apply the brakes but was suspected of watching a movie while the car was in Autopilot.

In this instance, it appeared enough evidence could be gathered to make such a determination. In the future, diagnosing why Autopilot or other self-driving algorithms made certain choices will likely only become more and more challenging as the algorithms rise in complexity.

At times, when I have my Tesla in Autopilot mode, the car will do something bizarre and I'll take over. For example, if I drive to work out of San Francisco, I have to exit left and merge onto the 101 using a ramp that arcs to the left almost 90 degrees. There are two lanes on that ramp, but even if I start in the far left lane and am following a car in front of me my car always seems to try to slide over to the right lane.

Why does it do that? My only mental model is the one I know, which is my own method for driving. I look at the road, look for lane markings and other cars, and turn a steering wheel to stay in a safe zone in my lane. But thinking that my car drives using that exact process says more about my limited imagination than anything else because Autopilot doesn't drive the way humans do. This becomes evident when you look at videos showing how a self-driving car "sees" the road.

When I worked at Flipboard, we moved to a home feed that tried to select articles for users based on machine learning. That algorithm continued to be to tweaked and evolved over time, trying to optimize for engagement. Some of that tweaking was done by humans, but a lot of it was done by ML.

At times, people would ask why a certain article had been selected for them? Was it because they had once read a piece on astronomy? Dwelled for a few seconds on a headline about NASA? By that point, the algorithm was so complex it was impossible to really offer an explanation that made intuitive sense to a human, there were so many features and interactions in play.

As more of the world comes to rely on artificial intelligence, and as AI makes great advances, we will walk to the edge of a chasm of comprehension. We've long thought that artificial intelligence might surpass us eventually by thinking like us, but better. But the more likely scenario, as recent developments have shown us, is that the most powerful AI may not think like us at all, and we, with our human brains, may never understand how they think. Like an ant that cannot understand a bit about what the human towering above them is thinking, we will gaze into our AI in blank incomprehension. We will gaze into the void. The limit to our ability to comprehend another intelligence is our ability to describe its workings, and that asymptote is drawn by the limits of our brain, which largely analogizes all forms of intelligence to itself in a form of unwitting intellectual narcissism.

This is part of the general trend of increasing abstraction that marks modern life, but it is different than not knowing how a laptop is made, or how to sew a shirt for oneself. We take solace in knowing that someone out there can. To admit that it's not clear to any human alive how an AI made a particular decision feels less like a ¯\_(ツ)_/¯ and more like the end of some innocence.

I suspect we'll continue to tolerate that level of abstraction when technology functions as we want it to, but we'll bang our heads in frustration when it doesn't. Like the annoyance we feel when we reach the limits of our ability to answer a young child who keeps asking us "Why?" in recursive succession, this frustration will cut deep because it will be indistinguishable from humiliation.

Evaluating mobile map designs

I saw a few links to this recent comparison by Justin O'Beirne of the designs of Apple Maps vs. Google Maps. In it was a link to previous comparisons he made about a year ago. If you're into maps and design, it's a fairly quick read with a lot of useful time series screenshots from both applications to serve as reference points for those who don't open both apps regularly.

However, the entire evaluation seems to come from a perspective at odds with how the apps are actually used. O'Beirne's focus is on evaluating these applications from a cartographic standpoint, almost as if they're successors to old wall-hanging maps or giant road atlases like the ones my dad used to plot out our family road trips when we weren't wealthy enough to fly around the U.S. 

The entire analysis is of how the maps look when the user hasn't entered any destination to navigate to (what I'll just refer to as the default map mode). Since most people use these apps as real-time navigation aids, especially while driving, the views O'Beirne dissects feel like edge cases (that's my hypothesis, of course; if someone out there who has actual data on % of time these apps are used for navigation versus not, I'd love to hear it, even if it's just directional to help frame the magnitude).

For example, much of O'Beirne's ink is spent on each application's road labels, often at really zoomed out levels of the map. I can't remember the last time I looked at any mobile mapping app at the eighth level of zoom, I've probably only spent a few minutes of my life in total in all of these apps at that level of the geographic hierarchy, and only to answer a trivia question or when visiting some region of the world on vacation.

What would be of greater utility to me, and what I've yet to find, is a design comparison of all the major mapping apps as navigation aids, a dissection of the UX in what I'll call their navigation modes. Such an analysis would be even more useful if it included Waze, which doesn't have the market share of Apple or Google Maps but which is popular among a certain set of drivers for its unique approach to evaluating traffic, among other things.

Such a comparison should analyze the visual comprehensibility of each app in navigation mode, which is very different from their default map views. How are roads depicted, what landmarks are shown, how clear is the selected path when seen only in the occasional sidelong glance while driving, which is about as much visual engagement as a user can offer if operating a 3,500 pound vehicle. How does the app balance textual information with the visualization of the roads ahead, and what other POI's or real world objects are shown? Waze, for example, shows me other Waze users in different forms depending on how many miles they've driven in the app and which visual avatars they've chosen.

Of course, the quality of the actual route would be paramount. It's difficult for a single driver to do A/B comparisons, but I still hope that someday someone will start running regular tests in which different cars, equipped with multiple phones, each logged into different apps, try to navigate to the same destination simultaneously. Over time, at some level of scale, such comparison data would be more instructive than the small sample size of the occasional self-reported anecdote.

[In the future, when we have large fleets of self-driving cars, they may produce insights that only large sample sizes can validate, like UPS's "our drivers save time by never turning left." I'd love if Google Maps, Apple Maps, or Waze published some of what they've learned about driving given their massive data sets, a la OKCupid, but most of what they've published publicly leans towards marketing drivel.]

Any analysis of navigation apps should also consider the voice prompts: how often does the map speak to you, how far in advance of the next turn are you notified, how clear are the instructions? What's the signal to noise? What are the default wording choices? Syntax? What voice options are offered? Both male and female voices? What accents?

Ultimately, what matters is getting to your destination in the safest, most efficient manner, but understanding how the applications' interfaces, underlying data, and algorithms influence them would be of value to so many people who now rely on these apps every single day to get from point A to B. I'm looking for a Wirecutter-like battle of the navigation apps, may the best system win.

    The other explicit choice O'Beirne makes is noted in a footnote:

    We’re only looking at the default maps. (No personalization.)
     

    It is, of course, difficult to evaluate personalization of a mapping app since you can generally only see how each map is personalized for yourself. However, much of the value of Google Maps lies in its personalization, or what I suspect is personalization. Given where we are in the evolution of many products and services, analyzing them in their non-personalized states is to disregard their chief modality.

    When I use Google Maps in Manhattan, for example, I notice that that the only points of interest (POI's) the map shows me at various levels of zoom seem to be places I've searched for most frequently (this is in the logged in state, which is how I always use the app). Given Google's reputation for being a world leader in crunching large data sets, it would be surprising if they weren't selecting POI labels, even for non-personalized versions of their maps, based on what people tend to search for most frequently.

    In the old days, if you were making a map to be hung on the wall, or for a paper map or road atlas, what you chose as POI's would be fixed until the next edition of that map. You'd probably choose what felt like the most significant POI's based on reputation, ones that likely wouldn't be gone before the next update. Eiffel Tower? Sure. Some local coffee shop? Might be a Starbucks in three months, best leave that label off.

    Now, maps can be updated dynamically. There will always be those who find any level of personalization creepy, and some are, but I also find the lack of personalization to be immensely frustrating in some services. That I search for reservations in SF on Open Table and receive several hundred hits every time, sorted in who knows what order, instead of results that cluster my favorite or most frequently booked restaurants at the top, drives me batty.

    When driving, personalization is even more valuable because it's often inconvenient or impossible to type or interact with the device for safety reasons. It's a great time saver to have Waze guess where I'm headed automatically ("Are you driving to work?" it asks me every weekday morning), and someday I just want to be able to say "give me directions to my sister's" and have it know where I'm headed.

    My quick first person assessment, despite the small sample size caveats noted earlier:

    • I know that Apple Maps, as the default on iOS, has the market share lead on iPhone by a healthy margin. Still, I'll never get past the time the app took me off to a dead end while I was on the way to a wedding, and I've not used it since except to glance at the design. It may have the most visually pleasing navigation mode aesthetic, but I don't trust their directions at the tails. Some products are judged not on their mean outcome but their handling of the tails. For me, navigation is one of those.
    • It's not clear if Apple Maps should have a data edge over Google Maps and Waze (Google bought Waze but has kept the app separate). Most drivers use it on the iPhone because it's the default, but Google got a headstart in this space and also has a fleet of vehicles on the road taking Google street photos. Eventually, Google may augment that fleet with self-driving cars.
    • I trust Google Maps directions more than those of Apple Maps. However, I miss the usability of the first version of Google Maps, which came out on iOS way back with the first iPhone. I'd heard rumors Apple built that app for Google, but I'm not sure if that's true. The current flat design of Google Maps often strands me in a state in which I have no idea how to initiate navigation. I'd like to believe I'm a fairly sophisticated user and yet I sometimes sit there swiping and tapping in Google Maps like an idiot, trying to get it to start reading turn by turn directions. Drives me batty.

    I use Waze the most when driving in the Bay Area or wherever I trust that there are enough other drivers using Waze that it will offer the quickest route to my destination. That seems true in most major metropolitans. I can tell a lot of users in San Francisco use Waze because sometimes, when I have to drive home to the city from the Peninsula, I find myself in a line of cars exiting the highway and navigating through some random neighborhood side street, one that no one would visit unless guided by an algorithmic deity. 

    I use Waze with my phone mounted to one of those phone clamps that holds the phone at eye level above my dashboard because the default Tesla navigation map is still on Google Maps and is notoriously oblivious to traffic when selecting a route and estimating an arrival time. Since I use Waze more than any other navigation app, I have more specific critiques.

    • One reason I use Waze is that it seems the quickest to respond to temporary buildups of traffic. I suspect it's because the UI has a dedicated, always visible button for reporting such traffic. Since I'm almost always the driver, I have no idea how people are able to do such reporting, but either a lot of passengers are doing the work or lots of drivers able to do so while their car is stuck in gridlock. The other alternative, that drivers are filing such reports while their cars are in motion, is frightening.
    • I don't understand the other social networking aspects of Waze. They're an utter distraction. I'm not immune to the intrinsic rewards of gamification, but in the driving context, where I can't really do much more than glance at my phone, it's all just noise. I don't feel a connection to the other random Waze drivers I see from time to time in the app, all of which are depicted as various pastel-hued cartoon sperm. In wider views of the map, all the various car avatars just add a lot of visual noise.
    • I wish I could turn off some of the extraneous voice alerts, like "Car stopped on the side of the road ahead." I'm almost always listening to a podcast in the background when driving, and the constant interruptions annoy me. There's nothing I can do about a car on the side of the road, I wish I could customize which alerts I had to hear.
    • The ads that drop down and cover almost half the screen are not just annoying but dangerous as I have to glance over and then swipe them off the screen. That, in and of itself, is disqualifying. But beyond that, even while respecting the need for companies to make money, I can't imagine these ads generate a lot of revenue. I've never looked at one. If the ads are annoying, the occasional survey asking me which ads/brands I've seen on Waze are doubly so. With Google's deep pockets behind Waze, there must be a way to limit ads to those moments where they're safe or clearly requested, for example when a user is researching where to get gas or a bit to eat. When a driver has hands on the wheel and is guiding a giant mass of metal at high velocity, no cognitive resources should be diverted to remembering what brands you recall seeing on the app.
    • Waze still doesn't understand how to penalize unprotected left turns, which are almost completely unusable in Los Angeles at any volume of traffic. At rush hour it's a fatal failure, like being ambushed by a video game foe that can kill you with one shot with no advance warning. As long as it remains unfixed, I use Google Maps when in LA. I can understand why knowledge sharing between the two companies may be limited by geographic separation despite being part of the same umbrella company, but that the apps don't borrow more basic lessons from other seems a shame.
    • I use Bluetooth to listen to podcasts on Overcast when driving, and since I downloaded iOS 11, that connection has been very flaky. Also, if I don't have the podcast on and Waze gives me an voice cue, the podcast starts playing. I've tried quitting Overcast, and the podcast still starts playing every time Waze speaks to me. I had reached a good place in that Overcast would pause while Waze spoke so they wouldn't overlap, but since iOS 11 even that works inconsistently. This is just one of the bugs that iOS 11 has unleashed upon my phone, I really regret upgrading.

    Things I learned from The Defiant Ones

    Despite believing myself fairly in tune with the pop culture scene, I missed a lot of promotion for The Defiant Ones until I started seeing recommendations on social media from folks who'd watched it. I finally blitzed through the four episodes recently, and it's kind of a banger.

    I typically don't love documentaries which comprise so many talking head interviews because it feels like the default Powerpoint template of documentary filmmaking. But Iovine and Dre and all the other musicians are such compelling, scene-filling personalities that it's a treat, and often a lark, to see them play to the camera. Allen and Albert Hughes interview all the principals individually, but as with all oral histories, they ask all of them about the same events so they can use shots from one interviews as a reaction shot to a shot from someone else's interview. Or as a reaction shot to historical footage, like Puff Daddy recalling his reaction to Suge Knight's acceptance speech at the 1995 Source Music Awards.

    In part, I was an easy mark because so much of that is the music of my youth. I was an intern at Procter and Gamble, living with a bunch of the other interns in a corporate apartment, the summer The Chronic came out. My roommates and I listened to that album just about every day, on loop, for no other reason than to mainline its hooks.

    The Defiant Ones is also fascinating as a case study of two immensely successful people, Jimmy Iovine and Dr. Dre. It is dangerous to draw too many conclusions from a documentary like this. Survivor and selection bias influence the narrative, and two people does not a large sample make. The mere act of narrative construction is a con game, and always will be, even when it isn't hagiography, which first person narratives like this always veer towards. So take the following with a Himalayan salt block, because I do.

    And yet...

    If I lump the stories in this documentary with what I know of other successful people, a few things stood out to me. Call this a Malcolm Gladwellian attempt at teasing out a few lessons from anecdotal evidence.

    The first is that people who are really good at what they do stand out from others by not only recognizing when something is exceptional immediately but articulating why it is so, especially when no one else believes it is. Designers experience this when they show a design to someone else, maybe a peer, maybe an executive, and that audience member immediately notices something the creator is particularly proud of. Stories of Steve Jobs moments like this abound, which is why everyone who has met Jobs even once seems to speak of it as some mystical experience.

    Filmmakers all have stories of screening cuts for others and having the sharpest among the notice a particular bit of directorial intent, maybe something in the choreography of the actors, or the camera's movement, or even something in the sound design, that no one else picks up on.

    Whether that pattern recognition is innate or trained over many years, and likely both, we see it again and again in The Defiant Ones. It's Jimmy Iovine cribbing "Stop Draggin' My Heart Around" from Tom Petty for Stevie Nicks. It's Iovine hearing Trent Reznor and fighting tooth and nail to grab Nine Inch Nails for Interscope from TVT Records. Or Iovine meeting Gwen Stefani and telling her she'd be a star in six years, and having No Doubt release Tragic Kingdom exactly six years after that conversation.

    [Remember my caveats up front? Steve Gottlieb of TVT records disputes the way the Nine Inch Nails story is framed in the documentary. I certainly don't think Iovine and Dre are the only ones in the music industry who possess this skill, but this documentary is their story so I'll roll with these examples for those who have or will watch the documentary.]

    The most memorable aha moment in the documentary, for me, is when Dre hears one bit from a demo tape from among hundreds of demo tapes stacked in Iovine's garage. 

    "Back in those days, I didn't have an artist to work with. I'd go to Jimmy's house, and we'd have listening sessions. He was trying to help me figure out where I was going to go with my music. He'd take me down to his garage. There was cassette tapes everywhere. And I remember him picking up this cassette tape. He pops it in. I was like 'What the fuck, and who the fuck is that?!"
     

    Who he was was an unknown white rapper from Detroit. In the documentary, in the recreation of that seminal moment, the label on the cassette tape reads Slim Shady. I'm not sure if that's actually true to history, but it's remarkable both ways. In one, it's a wonderful bit of historical trivia, in another, it's a laughably on the nose historical recreation.

    Again, we have this pattern, the flash of recognition, picking out this tape from all the demo tapes, and hearing what no one else heard. With things like music, or even food, the articulation of excellence isn't as critical as the recognition. As in the excerpt above from Dre's memory of that moment, it was probably just a series of expletives, perhaps a literal WTF as he recalls.

    The moment where Dre recognizes the kid's talent isn't online, but this clip from Eminem and Dre's first meeting is, and it's amazing because it contains footage of the end from their first session in the studio. The tail end of this clip reveals what happened when Dre started playing a few beats he was working on for Eminem, and it's gobsmacking because so rarely is the moment of creative conception captured on video. See for yourself.

    "Like yo. Stop. Shit's hot. That's what happened our first day, in the first few minutes of us being in the studio," remembers Dre.

    Because Eminem was a scrawny white rapper from Detroit, many resisted. He didn't look the part. That brings up the second lesson.

    "My gut told me Eminem was the artist that I'm supposed to be working with right now," Dre recalls. "But, I didn't know how many racists I had around me."

    "Everybody around me, the so-called execs and what have you, were all against it. The records I had done at the time, they didn't work, they wanted me out the building. And I come up with Eminem, this white boy."

    As in many moments in their long collaboration, Iovine and Dre persisted and profited yet again by arbitraging the biases of the herd.

    "We weren't looking for a white, controversial rapper," Iovine says. "We were looking for great."
     
    "Great can come from anywhere."
     

    He means it.

    "Lady Gaga walked into my office, Italian girl with brown hair, started telling me about Andy Warhol, and dance music, but yet industrial, and paintings. I don't know, she confused me so much that I signed her."
     

    None of the other Interscope execs thought Gaga had breakout appeal. Iovine did.

    "I was at a club with Timbaland, and I saw the room move. It felt like pop music. It felt like it could break through."
     

    Perhaps not a snap judgment, but no one would confuse Lady Gaga for Eminem. When Iovine says great can come from anywhere, his diverse roster of artists backs him up.

    How do you find alpha in an otherwise efficient market? Iovine and Dre arbitraged the biases of the market, of which one is rampant pattern recognition.

    Much of this makes it sound as if identifying hit music is Iovine and Dre's talent. But plenty of evidence exists that much of cultural taste is socially constructed and is subject to path dependence.

    The truth, as always, is somewhere in the middle. However, most people underestimate how much It is possible to socially hack popularity since some of popularity is a social construction and has nothing to do with any inherent quality of the goods being sold. This is a third lesson which the documentary reinforces.

    Derek Thompson's Hit Makers, which I will write up soon, and Michael Mauboussin's The Success Equation, both of which I loved, both cite Duncan Watts and Matthew Salganik's MusicLab experiments. The key finding of that study was that people rank some things higher simply because they were given randomly generated hints that those things were already popular with other people.

    This Kevin Simler post offering an alternative explanation for how ads work is actually how many people who work in advertising understand ads to work, at least in part. Simler theorizes that ads create common knowledge, and much as Watts and Salganik's experiment reveals, so much of human behavior is socially constructed. In the case of the MusicLab study, it's popularity. In Simler's examples, ads cue consumers on which products are likely to be the most effective signals in a world which status is socially constructed in large part through such consumer product totems.

    Iovine understands this, and nowhere is it more evident than in the latter part of the final episode of Defiant Ones, the Beats Headphones saga. 

    Dre spends some lots of time engineering Beats headphones for a particular sound. More on that later. I'm a moderate headphone geek; enough so that I own more than four pairs of over ear headphones (I prefer the sound of some for specific types of music) and two headphone amps, so I appreciate what Dre understands, which is that the personality of headphones can be a matter of personal taste.

    Iovine cuts to what's far more important in the headphone decision. Most people don't give a hoot what the response curves of a headphone are measured at, what they sound like. People wear them as fashion accessories, and people want to be cool.

    Iovine and Dre set up a day where they test all the leading headphones on the market. They're not impressed.

    "We realized that all headphones sound boring and looked like medical equipment. We wanted more bass in these headphones to exaggerate all of it. We wanted to put it on steroids," Iovine said.
     
    Producer Jon Landau recalls: "The Bose headphones, they were advertising noise canceling, total quiet. Jimmy says, 'Noise canceling?! Yeah, they're the headphones if you want to go to sleep on a plane. Our headphones are the where's the party headphones.'"
     

    The distribution and marketing leverage was to be found through Iovine's celebrity friendships, so he starts smiling and dialing, or perhaps more appropriately in Iovine's case, dialing and cajoling. He gives those headphones away to all his artists and asks them to wear them in their music videos, in public, anywhere a camera or a human eye is present. Anyone famous walking in Iovine's office has to don a pair of the headphones and submit to a photo. The design of the Beats headphones, like the iconic white headphones for the iPods, is brilliant. The iconic b imprinted on each colorful molded plastic ear cup is like a walking billboard.

    After artists, Iovine moves onto athletes, and soon it's rare to see Lebron benching in any of his workout videos on Instagram without his Beats by Dre headphones. I almost can't picture Ronda Rousey walking into the ring or out of the ring without picturing her with her Beats headphones draped around her neck. Who can forget Michael Phelps staring down Chad Le Clos in the 2016 Olympics, his Beats headphones blasting what must surely be some angry heavy metal that would ripple the surface of the Olympic pool.

    All PR isn't good PR, but when the sports leagues like FIFA and the NFL and the Olympics issue bans on the Beats headphones, it's a dream come true for a product seeking renegade cachet.

    It works. Any self-respecting audiophile considers Beats to be an absolute scam from a sound quality perspective and yet Beats dominates the premium headphone ($99 or greater) market.

    Not every product market sees market share driven by socially constructed popularity, but headphones are perhaps the perfect fashion accessory and cultural signal in the age where everyone can listen to music through their smartphone at any time.

    Iovine pushes the headphones so much that Eminem admits it annoyed him.

    "There would be times where we would be shooting a video until like six in the morning, and we had to do one more take with me or somebody in the video wearing some goddamn [Beats] headphones. Are you fucking kidding me?!"
     

    Iovine is a great producer, but he's also a consummate marketer.

    "The only person that does it better than him is me," says Puff Daddy.
     

    There may be a line which is shameful to cross when it comes to marketing, but who knows where that line is if you have no shame.

    "He's got good instinct, and he's shameless," says Trent Reznor about Iovine.
     

    In fairness to the documentary, Dre does talk a lot about tuning the sounds of the Beats headphones, so why do audiophiles dislike the sound? Beats are notoriously bass heavy. Dre grew up listening to music in cars in LA, with subwoofers so heavy that people outside the car can feel their organs being jostled.

    Music, especially for young people, is raw emotion and energy. Not that audiophiles don't also love to turn up their music, but the bass-heavy sound Dre and Iovine amplifies the primal elements of the music, something that non-audiophiles can feel. In a revealing scene, Dre demos the mix of an album by taking Iovine to a garage to listen to the album in a tricked out van. Dre knows that the music of the street is often heard, literally, on the streets, coming through some car stereo, bass pumping, car rocking. Dre isn't above understanding the social transmission of music, it's just that he understands a particular form of that virality, when it comes through the original social network, the streets of the neighborhood. If it weren't likely to render its listeners deaf, Dre would probably want his headphones to sound like those cars which wake the neighborhood, the bass so powerful that the subwoofers seem to shake windows and cause a car to bounce up and down.

    The last bit, which is a meta point, and one I've been thinking about a lot recently, is how many more entrepreneurs The Defiant Ones will reach and teach than any single book on entrepreneurship. Video may be a lossy medium in terms of how much it leaves out in service of the narrative structure, but its inherent visual and "autoplay" quality are proven to be much lower friction as an educational medium than text. We need more like this and less like the typical MOOC video which replicates all the excitement of your median classroom lecture.

    When you come to the 2^100 forks in the road...

    In some simple games, it is easy to spot Nash equilibria. For example, if I prefer Chinese food and you prefer Italian, but our strongest preference is to dine together, two obvious equilibria are for both of us to go to the Chinese restaurant or both of us to go to the Italian restaurant. Even if we start out knowing only our own preferences and we can’t communicate our strategies before the game, it won’t take too many rounds of missed connections and solitary dinners before we thoroughly understand each other’s preferences and, hopefully, find our way to one or the other equilibrium.
     
    But imagine if the dinner plans involved 100 people, each of whom has decided preferences about which others he would like to dine with, and none of whom knows anyone else’s preferences. Nash proved in 1950 that even large, complicated games like this one do always have an equilibrium (at least, if the concept of a strategy is broadened to allow random choices, such as you choosing the Chinese restaurant with 60 percent probability). But Nash — who died in a car crash in 2015 — gave no recipe for how to calculate such an equilibrium.
     
    By diving into the nitty-gritty of Nash’s proof, Babichenko and Rubinstein were able to show that in general, there’s no guaranteed method for players to find even an approximate Nash equilibrium unless they tell each other virtually everything about their respective preferences. And as the number of players in a game grows, the amount of time required for all this communication quickly becomes prohibitive.
     
    For example, in the 100-player restaurant game, there are 2&100 ways the game could play out, and hence 2^100 preferences each player has to share. By comparison, the number of seconds that have elapsed since the Big Bang is only about 2^59.
     

    Interesting summary of a paper published last year that finds that for many games, there is not clear path to even an approximate Nash equilibrium. I don't know whether this is depressing or appropriate to the state of the world right now, it's probably both. Also, it's great to have mathematical confirmation of the impossibility of choosing where to eat when with a large group.

    Regret is a fascinating emotion. Jeff Bezos' story of leaving D.E. Shaw to start Amazon based on a regret minimization framework is now an iconic entrepreneurial myth, and in most contexts people frame regret the same way, as something to be minimized. That is, regret as a negative.

    In the Bezos example, regret was a valuable constant to help him come to an optimal decision at a critical fork in his life. Is this its primary evolutionary purpose? Is regret only valuable when we feel its suffocating grip on the human heart so we avoid it in the future? As a decision-making feedback mechanism?

    I commonly hear that people regret the things they didn't do more than the things they do. Is that true? Even in this day and age where one indiscretion can ruin a person for life?

    In storytelling, regret serves two common narrative functions. One is as the corrosive element which reduces a character, over a lifetime of exposure, to an embittered, cynical drag on those around them. The second is as the catalyst for the protagonist to make a critical life change, of which the Bezos decision is an instance of the win-win variety.

    I've seen regret in both guises, and while we valorize regret as life-changing, I suspect the volume of regret that chips away at people's souls outweighs the instances where it changes their lives for the better, even as I have no way of quantifying that. Regardless, I have no contrarian take on minimizing regret for those who suffer from it.

    In that sense, this finding on the near impossibility of achieving a Nash equilibrium in complex scenarios offers some comfort. What is life or, perhaps more accurately, how we perceive our own lives but as a series of decisions, compounded across time.

    We do a great job of coming up with analogies for how complex and varied the decision tree is ahead of us. The number of permutations of how a game of chess or Go might be played is greater than the number of atoms in the universe, we tell people. But we should do a better job of turning that same analogy backwards in time. If you then factor in the impact of other people in all those forks in the road, across a lifetime, what we see is just as dense a decision tree behind us ahead of us. At any point in time, we are at a node on a tree with so many branches behind it that it exceeds our mind's grasp. Not so many of those branches are so thick as to deserve the heavy burden of regret.

    One last tidbit from the piece which I wanted to highlight.

    But the two fields have very different mindsets, which can hamper interdisciplinary communication: Economists tend to look for simple models that capture the essence of a complex interaction, while theoretical computer scientists are often more interested in understanding what happens as the models grow increasingly complex. “I wish my colleagues in economics were more aware, more interested in what computer science is doing,” McLennan said.

    Selfies as a second language

    Of the people I follow on Snapchat, about half are old people (lots of middle-aged white male VC's, maybe trying to make sense of what it is), the other half are young and what I'd consider Snapchat natives. As a product person, it's fascinating to observe stark divides in consumer behavior. Often these are generational divides, less discussed than technological shifts like those fueling platform shifts or industrial revolutions, but no less fascinating. Often, these behavioral changes happen because of technology shifts, as humans evolve with their new tools and altered environment. In Snapchat is one of the cleanest, most universal of these behavioral fault lines.

    When I send a Snap to any of the people in my address book, the oldies respond, inevitably, with some text message, maybe an emoji if they're somewhat hip. If I send a Snap to a young'un, inevitably I'll receive a selfie in response.

    Since I noticed this a few years back, I've tracked it across the years, and it's still the case, to an astonishingly consistent degree. I'm talking nearly 100%, and I can't remember any exceptions.

    My theory on this is that older folks did not grow up with front facing cameras on smartphones and thus experience an uncomfortable body alienation from seeing themselves in photographs akin to how most people hate hearing their own voices the way other people hear them. We perceive our own voice differently than others because we hear our own voices reflected back from the world mixed in with feedback from the machinery we use to generate that sound. Other people only hear the former.

    For my generation, we grew up mostly seeing ourselves in mirrors, and thus that's the way we visualize our face and body. When we see ourselves in photos, we see a flip of what we're used to seeing in the mirror, and it's discomfiting.

    Cameras can introduce additional distortion depending on the focal length of the lens. Almost all smartphone cameras have wide-angle lenses. The iPhone camera is something like 28mm (in 35mm camera terms); I'm not sure what the front-facing camera is, but it's wide. A 50mm focal length is typically considered a neutral lens in 35mm cameras, and any focal length shorter than that is usually considered wide. 

    That old cliche about how the camera adds ten pounds? It refers to the distorting effect of wide angle lenses which are very common in television and film, especially for a lot of closeups and medium shots. If you ever see an actor or model in person they look surprisingly thin. People who look normal on camera look thin in person, and models, who look thin on camera, look malnourished in person.

    What is unpleasant for faces can be flattering for spaces. Almost all housing interiors for sites like Airbnb or any real estate listing are shot with wide angle lenses. Often it's the only way to capture an entire room from a photo shot within the room, but it has the pleasant effect of expanding the space. This is why, if you ever go to a live taping of a TV show like Saturday Night Live or The Daily Show, it's shocking how tiny the studio actually is compared to how it appears on TV, and why that seemingly spacious apartment you rented on Airbnb feels like a bathroom stall when you arrive in person, roller bag in tow.

    If the wide angle smartphone camera lens renders people's faces larger, the effect is exaggerated when the camera is held at mere arm's length. It makes people look heavier than they're used to seeing themselves in the mirror, and that's not pleasant for all except those with tail distribution positive body image. There's a reason a portrait lens is usually longer than neutral, often starting at 85mm or longer, and why fashion shoots often use telephoto lenses that require a photographer to stand really far from their subjects, sometimes so far they have to shout directions to the model through a megaphone. The longer the lens, the shallower the focus, the more flattering the portrait.

    However, this generation of kids who've grown up with a smartphone pointed at their faces from the time they were infants have seen themselves hundreds if not thousands of times through the funhouse mirror eye that is the smartphone camera. So much so, I speculate, that they experience a much lower degree of photographic body alienation than my generation. I may see myself in a bathroom mirror a few times a day, once in the morning, a few times at work, and once at night. Kids of this generation, armed with smartphones from an early age, often see themselves in photographs, on a screen, dozens of times a day.

    Furthermore, they've internalized this disparity in impact between their photographic and real world representation the way celebrities and models do. It's just math. Ubiquitous smartphones and social media allow exponentially more people to see their photographic self than their real world body. It's entirely rational to consider their virtual self to be more important in the accumulation of social capital than their physical selves. Time spent mastering the selfie is time spent on the largest audience, and while it may be horrifying to see young people shooting dozens of photographs of themselves before posting to social media, then subsequently A/B testing which photos garner the most positive social media feedback, it's behavior one would predict for homo socialis.

    They say a picture is worth a thousand words. While the exact ratio may be something we can actually calculate with a cleverly designed experiment, even in the absence of such a test it's clear that the multiplier is significant. We are, all of us, straining against the more narrow emotional register of text, especially when we often face character limits, both imposed and self-inflicted (due to the inconvenience of typing on smartphone keyboards). As email and text messaging replaced more expressive mediums like phone calls and handwritten letters, we find ourselves apologizing, quite often, for coming across other than we intended.

    It's no surprise that many writers resorted to adding emotional modifiers like =) to emails. Even prior to those early emoji, the use of exclamation points in online communication was noticeably higher than in regular writing, lest we come off as bored, or even worse, cold (increasingly, it becomes a contrarian power move to eschew exclamation points entirely; the irony of Donald Trump ending every tweet with an exclamation point is how silly it is for the leader of the free world to resort to such linguistic chest puffery).

    After early emoticons came the age of emoji, and now the GIF has shouldered its way into the conversation. Each successive communications trend brings a more efficient carrier of emotion, per byte, than text, and compression matters in this clipped conversational age.

    In most cases, I much prefer receiving a selfie in reply to a message I send than a text response, because the human face is a miraculous instrument, almost incapable of the abstraction of raw text. Still, I can't bring myself to send selfies as responses.

    Perhaps if I looked like Ryan Gosling or Gal Gadot, I'd spend hours admiring myself in a mirror, snapping selfies at all different angles just to see if it was even possible to make myself look anything less than gorgeous. Is it even possible for Denzel Washington to cringe at the sound of his own voice played back to him? If I were Denzel I'd just talk to myself all day, just to marvel at how I could make anything sound like the word of God.

    But I suspect it's more than that. I've happily embraced emoji and the fetishistic allusiveness of the GIF. When it comes to the selfie, however, I'm a not-yet adopter. I am of that generation for whom selfies are not second nature but instead a second language.