The secret technology of The Daily Show

Many have mourned Jon Stewart's announcement that he'll be leaving The Daily Show this year. Count me among those dressed in black; Stewart felt like my cool, whip smart Jewish uncle the past 16 years. One can claim that sometimes it's the format of a program that endures, and not the bodies filling the seats—for example, with Saturday Night Live—but with both The Daily Show with Jon Stewart and The Colbert Report, that's just wishful thinking. These two shows, like the late night talk shows, have long had their hosts very names in the titles, and for good reason; without Stewart and Colbert, the shows will become something different out of both necessity and circumstance.

Emily Nussbaum wrote a wonderful appreciation of Stewart's legacy, and one piece of it caught my eye for pointing out what I consider the show's most undervalued skill.

The truth is that Stewart was often at his most exciting when he got down in the dirt, instead of remaining decent and high-minded, your twinkly-eyed smartest friend. Five years ago, when he confronted MSNBC’s financial reporter Jim Cramer over his coverage of Wall Street, Stewart refused to be collegial. He nailed Cramer on his manipulations, airing clip after damning clip, and shouting “Roll 212!” with prosecutorial glee. He was a good interviewer with people he admired, but in some of the show’s most memorable segments he relied on search technology—in particular, his staff’s ability to cull clips and spin them into brutal montages—to expose lies that might have gone unremarked upon. Over time, he became not merely a scourge of phonies but the nation’s fact checker, training others in the craft. You can see that influence not only among hosts who started out on “The Daily Show,” including Colbert, John Oliver, and Larry Wilmore, but everywhere online. Twitter, on its best nights (and they do exist, doubters), can feel like a universe of sparkling mini-Stewarts, cracking wise but also working to mob-solve the latest crisis, and providing access to a far wider array of perspectives than any one comic could.

That kind of digging, of disrespecting authority, was a model for reinventing journalism, not comedy.

The secret technology behind The Daily Show was search.

Any viewer is, by now, familiar with the show's format. The opening third, almost always my favorite, would feature Stewart tackling a variety of the most prominent current events in politics and society and putting either some of the protagonists or the media on trial. Sometimes he'd dissect them himself, like a gifted if somewhat smug trial lawyer, but more often than not, he won by jiu jitsu. He let witnesses hang themselves on a rope of their own words.

I've never read how they do it, but the Daily Show seems to have catalogued every piece of video from every politician and reporter in the history of television. Did a politician claim one thing? Here's a clip of them from another time, contradicting themselves. Did Fox News castigate Obama for his decisiveness on a piece of foreign policy? Here's a clip of their anchors praising Bush for the same quality when it came to a similar situation. Often that opening portion of The Daily Show felt like a Three Stooges clip, with hapless politicians slapping themselves in the face, Stewart and his writing staff pulling the strings.

Do they have banks and banks of cable boxes and DVRs, recording every minute of CSPAN, Fox News, CNN, and MSNBC, converting all the dialogue to text, labeling every moment with row after row of metadata? How many researchers do they have on staff? How do they retrieve clips so quickly each day, and what is the interface for that system? Can they run searches by simply stringing together words like "Bill O'Reilly" "hypocrisy" "Iraq War"? Or is there a giant dropdown box with a bunch of predefined categories like "old white senators saying racist things"?

In turning what seems like the entire history of televisions news into a deeply catalogued primary source, The Daily Show lifted the journalistic standard of television news. This isn't a new phenomenon. The internet is, above all else, the greatest information distribution technology in history, and many a writer or journalist has realized too late that it's not their immediate fact checker or editor whose standards matter but that of millions of internet-connected people with lots of time and Google as their default homepage. Linus Torvalds is credited for saying “given enough eyeballs, all bugs are shallow.” I propose a corollary, “given enough eyeballs and enough metadata, all lies become public.”

In cycling, drug testing authorities keep samples of bloods for years after events so they can test samples retroactively as better drug-detection tests are devised. Why, in the age of the internet, people continue to plagiarize is beyond me, but even if one can get away with it for the moment, everything ever written and posted online lives on until that day when the original text is indexed and made searchable and detecting the crime becomes a matter of a trivial exact match query.

Video is late to this game, though, because it's much harder to index the spoken dialogue in video. Some companies have solutions, I've seen many a demo at trade shows, and we indexed closed caption files at Hulu, too. However, it's still not easily available to consumers on a significant percentage of video online. Yet. That's what made what I'm presuming to be The Daily Show's video catalog or index so remarkable.

The third episode of Season 1 of Black Mirror, “The Entire History of You,” postulated a world in which The Daily Show's technology for trapping people with evidence of their own hypocrisy existed in our personal lives. An implant in our brains would record and index every moment of our lives, allowing us to put each other on trial for the rest of our days. It's a common downside scenario for total recall technology, mentioned in almost any article that has experimented with   prototypes.

That episode of Black Mirror ends badly, as is common in this age of somewhat bleak science fiction. Real world evidence isn't so conclusive yet. Despite the almost nightly prosecution of The Daily Show with Jon Stewart, politicians and media like Fox News don't seem to have changed their behavior much, at least not to any level I can detect. Not even the rich and powerful are above shame, but it's safe to say many of them have a higher than average tolerance.

As for having our personal hypocrisies made shallow, I can't imagine that a greater leniency towards each other wouldn't win out over continual witch hunting. Furthermore, a mutually assured destruction of reputation might naturally result in a bottoms up detente. After all, who among us hasn't said something we later wished to expunge or walk back? Some people point to internet trolling as a counter-example, but I suspect it's largely over-indexing on the loud minority over the reasonable silent majority as our human brains love to do.

Even if such technology were widespread and forced us all to be more considered before we wrote or spoke, is that so bad? Taken to an extreme, that's a terrifying Orwellian scenario, but when Nussbaum writes that “[Stewart's] brand was decency,” she understands that much of the show's appeal was his own reasonable nature. Stewart often seemed exasperated at the rigid rhetorical stances in American politics, but it's difficult to believe he would have lasted 16 years at the desk if he didn't believe, deep down, that if we just hold up a mirror to ourselves, not a black mirror, nor one one ringed with flattering warm lights, but just the clearest one available, we'd grow the hell up.