Human and computer curation in the age of information abundance

I don't usually talk shop here, but I wanted to spend a few words discussing some of the key strengths of the new version of Flipboard we just shipped, v3.0 on the odometer. Though I've only been at the company just over a year and a half, Flipboard has been around since just after the first iPad shipped. In many ways, it's the first iPad app I can remember because it was the first native iPad app, the first app that didn't seem like a port of an iPhone app. It launched at first just for the iPad, and it wasn't until later that the a version of Flipboard for the phone shipped.

Because of its long history, so many of my readers may have played with Flipboard at some point in the past and forgotten about it. Many of my readers or Twitter followers are what I consider to be among the top 1% of voracious information consumers, and for them, high density information presentation, typically an RSS reader with hundreds of RSS feeds, or Twitter, with its neverending supply of links and headlines and thoughts, are the weapons of choice. I can understand that, and I use both as well.

But both of those options offer a certain inherent level of noise because of how they work. On Twitter it's because people you follow have lots of interests, some of which match yours, some of which don't. Sometimes they point you at things outside your sphere of interests, but once you're following several hundred people, as I am, the multitude of noise can pile up.

The RSS feed junkie is sort of the predecessor to the Twitter information junkie, and I know people who subscribe to literally hundreds of RSS feeds in a feed reader and try to keep up with the flood of headlines each day. The problem is the same there; it's almost inevitable that a list of that many information sources will inject a lot of perceived junk into your mental diet.

Though it doesn't feel like it to the top 1% of information addicts, cultivating a list of hundreds of people to follow or hundreds of RSS feeds is exhausting and tedious for the 99%. Keeping up is its own burden; I long ago gave up trying to read every tweet in my Timeline, or every headline from every RSS feed I follow.

At the same time, FOMO is a very real phenomenon not just in the real world but in the information space, and it's a natural outgrowth of the internet, the most efficient delivery mechanism of data that has ever been invented in the history of mankind. Effectively, there is a near infinite amount of information out there, and more is being generated each day than one person could read in a lifetime. Supply is no longer the problem; oversupply is.

Given that we all have a finite amount of attention to give each day, how do we allocate it most efficiently? Flipboard 3.0 is different strategy for answering that problem, and it's different not only from the other apps/services outlined above but even from past versions of itself. For that reason, it has something to offer both the power information consumer and the casual information grazer.

The new Flipboard, designed specifically for phones and built in part off of the great technology acquired with our purchase of Zite, centers around a person's interests. The combination of any person's interests forms a sort of intellectual fingerprint, and that is the North star for personalization in the app. You might find it challenging to curate a list of all the experts in a field across their blogs, websites, research papers, Twitter and Facebook accounts, etc. Multiply that across all of your interests, and the task of following the right sources and people grows by leaps and bounds. I follow nearly 900 people on Twitter and I'm still adding and deleting people all the time.

The new Flipboard simply asks for your interests, a very finite and manageable list, and then uses that to find the best articles for you on those topics, wherever those might reside, and regardless if you know or follow the authors or blogs. We offer over 30,000 such topics to choose from, and if you have no idea how to know if that's a good number, I can assure you it gets deep into the long tail. And it's growing all the time; when we started work on this release, there was no topic in our database called “Peter Piot”. We hadn't seen enough recent articles of note about Piot. And then a two year old boy died in West Africa, and the world changed. Peter Piot helped discover Ebola. New topics arise every day; a few weeks ago, we added “Apple Pay”, and soon we may have to add CurrentC, though maybe that will disappear before it has a chance to become a thing.

It's in the discovery of content on these tens of thousands of topics where computers and algorithms have a huge advantage over humans. You might think of Flipboard as an intelligent agent, an AI that can read and screen millions of new articles written each day, something no human can come close to doing. This intelligent agent knows what you're interested in and understands when it finds an article on that topic, and it can do that for millions of users on tens of thousands of topics across millions of articles every day.

However, to say human have been rendered obsolete in yet another field couldn't be further from the truth. It's of little use if your digital reader brings back every article on every topic you care about. That's still way an information deluge. Remember, the goal here is efficient allocation of your limited attention. Your intelligent agent also needs to separate the signal from the noise.

This is where humans remain in play. It turns out computers are good at determining the topic of article but not the quality. To do that, we turn to the aggregate reading and curation activity of millions of readers. The advantage of some scale in this space is that there are enough eyeballs staring at enough items of interest that you need only follow the collective gaze to find what's interesting. This is especially true with content that isn't based on text, like photographs, where computer vision is still fairly crude in comparison to human vision when it comes to understanding both subject matter and aesthetic value. Ask any social service of scale if they'd trust a computer to moderate content for pornography and other disturbing imagery; well over a 100,000 workers worldwide filtering the filthy and the horrific from our social streams are proof they don't.

[By the way, even with textual content, computers aren't perfect. A topic like “magic” is a good example. Computers will bring back content on the Orlando Magic, card tricks, and articles with headlines referring to Roger Federer as a magician with a tennis racket.]

Just as the best chess players in the world are not grandmasters or top computer programs but some combination of human player with a chess program, the best curation is still some combination of human judgment and a variety of computer algorithms. Someday perhaps there will be some digital general intelligence of such power that human taste is superfluous, but today is not that day.

Even all of this is not enough, though. It turns out that if you give every person exactly all these interesting articles on all the topics they tell you they want to see, they get bored. People are always chasing after the unexpected delight, it gives them a mental rush to stumble across a fascinating article they wouldn't have expected us to realize they'd like. This is the serendipity that is habit-forming.

Some argue that the serendipity can consist of just noise. That presumes people enjoy sifting through the irrelevant to find the gold dust. I think some people enjoy the hunt, but many are not so persistent, so our goal is for even the serendipity in Flipboard to be of interest. There are many approaches to achieving this, and I won't delve into the technology behind it all, it's a lot of math. All that matters for readers is that it works, because when it does it feels like magic.

One last thing: a person's interests evolve over time. You may choose a bunch of topics in your initial setup, but even if you don't alter that list again, it's possible to tell, based on your reading behavior, when your tastes, or what we call your affinities, change. The more you read and like and curate on Flipboard, the more your Flipboard will start to feel like a pair of raw denim jeans that break in and mold to your legs, or like a sportcoat or dress that's being tailored for you a bit each day.

My recommendation for both folks who may have used Flipboard before but haven't used it in a while and for loyalists: update to the new app, version 3.0 for those who still care about such designations, and follow a bunch of topics, and then follow even more, the more specific the better. At last count, I was following 90 topics on Flipboard, and that number has been growing by a few each week I've been playing and testing the app.

Here are just a few topics I'm following to give you a sense of the breadth of our database:

  • Joe Maddon - soon to be announced as the new Cubs manager, I hope.
  • Nikon D750 - I haven't upgraded my Nikon digital SLR for several generations, but I changed my mind for this baby. Finally, Nikon adds wi-fi so I can get a photo from my SLR to my phone quickly. Why they didn't add this years ago still boggles my mind.
  • Peter Thiel and Marc Andreessen - interesting thinkers in the technology world
  • Paul Thomas Anderson
  • Driverless Cars
  • GIF Animations - we should probably just rename this to Animated GIFs, but maybe this is how they're referred to in high company.
  • Augmented Reality
  • E-learning
  • David Foster Wallace
  • Information Design
  • Cinematography
  • Roger Deakins - speaking of cinematography...Deakins should've won an Oscar in 2008 for The Assassination of Jesse James by the Coward Robert Ford. Got a double nomination in that category that year! The other was for No Country for Old Men
  • Neuroscience
  • Cycling
  • Sports Analytics
  • Technological Singularity
  • Optical Illusions
  • Memes
  • Sneakers
  • Tesla
  • Elon Musk - the entrepreneur whose courage and bravado are the envy of the Valley.
  • Chicago Cubs - finally, a light at the end of the tunnel. Fellow Cubs fans, it's about to get good.
  • Interior Design - I finally bought a condo in June, I'm still in the midst of renovation and decoration hell, someone tell me it gets better.

One thing I've often felt in trying lots of services like Nuzzel and Quibb on top of Twitter and RSS Readers is seeing the same set of articles referred to me several times. I'll see it first appear on Twitter, then a short while later on services like Nuzzel or Quibb, and then a day later in emails of top tweets from Twitter, and then even later on some blogs I follow.

What was always refreshing about throwing Zite and now Flipboard into the mix was the sense of discovering things I hadn't already seen many times. It has taken a lot of tuning and testing to dial that in, and it's an ongoing project, but when it works it feels like magic, and it gives me the feeling of being ahead of the herd in finding things to link to on my blog or to post on Twitter.

Okay, I've stepped off my work soapbox. I'm biased, of course, so don't take my word for it, listen to Farhad Manjoo at the NYTimes. Or if you don't want to listen to Farhad, take the word of Jennifer Garner, who revealed her favorite app is Zite, whose core technology now powers key pieces of the new Flipboard.

While Garner may not have a big social media presence, she did reveal her favorite app. "Zite," she said. "It's like a magazine and it sends you all your favorite things. So mine are West Virginia, world news, kids, parenting, relationships, healthy living, food and cooking…It just curates exactly what I want to read."

She is definitely way more beautiful than I am (I'm not sure about Farhad, who I've never met), and with kids to raise and a Hollywood career, she's also busier and has a higher opportunity cost for her time, so if we can earn her trust in allocating her minimal free time, maybe there's something there.