Hackers

Interview with Arthur Chu, famous now for studying Jeopardy carefully and developing a strategy to optimize his chances to win. Some are criticizing his playing style, accusing him of hacking the system, but isn't that how one should try to play every game? Learn the rules and develop the optimal strategy?

I noticed some interesting things. Everyone's talking about my strategy on the show, it seems, but I didn't make anything up—I just read people's observations online. In 1985, the second year of the Alex Trebek version of the show, this guy Chuck Forrest really dominated by bouncing around the categories, and they call it the "Forrest Bounce." There's no logical reason to do what people normally do, which is to take one category at a time from the top down. Your only point of control in the game is your ability, if you get the right answer to a question, to select the next question—and you give that power up if you make yourself predictable. The more unpredictable you are, the more you put your opponents off-balance, the longer you can keep an initial advantage. Multiple people over the years have used [the Forrest Bounce] and yet most people haven't used it. When they programmed the computer Watson to maximize its chances of winning, it did the Forrest Bounce. And it specifically did Daily Double hunting. Watson knew that the bottom two rows of the Jeopardy! board are more likely to contain Daily Doubles, and it knew that whoever gets the most Daily Doubles is the most likely to win the game—that's just statistical analysis. So it was programmed to hunt those Daily Doubles, and I figured I had no reason not to do that.
 

Another excerpt of interest:

There are a few specific composers they want [you] to know. If they mention "a Norwegian composer"—this happened in a game, I think the Wednesday game—it will be Edvard Grieg. That's the method they use to write the clue. If they mention a "Polish Nobel Prize Winner," it's likely to be Marie Curie. If they mention a "Female Nobel Prize Winner," it's very likely to be Marie Curie. Jeopardy! is aimed at the sort of average TV viewer, so they're not going to ask things that are pointlessly obscure, they're not going to go in-depth on any particular subject, they're going to focus on these cultural touchstones that we all know.And if you watch the show, and you can identify those, you can literally make flashcards.

So I used a program called Anki which uses a method called "spaced repetition." It keeps track of where you're doing well or poorly, and pushes you to study the flashcards you don't know as well, until you develop an even knowledge base about a particular subject, and I just made flashcards for those specific things. I memorized all the world capitals, it wasn't that hard once I had the flashcards and was using them every day. I memorized the US State Nicknames (they're on Wikipedia), memorized the basic important facts about the 44 US Presidents. I really focused on those. But there's a lot more stuff to know. I went onJeopardy! knowing that there was stuff I didn't know. For instance, everyone laughs about sports—but I also knew that [sports clues] were the least likely to come up in Double Jeopardy and Final Jeopardy and be very important. So I decided I shouldn't sweat it too much, I should just recognize that I didn't know them and let that go, as long as I can get the high value clues.
 

*****

Gamifying world news instruction. Though gamification has some negative connotations thanks to many mobile games and online dark patterns of UI design, so perhaps we should refer to this as some variant of a nudge?

*****

The guy who hacked OkCupid to find love.

But mathematically, McKinlay’s compatibility with women in Los Angeles was abysmal. OkCupid’s algorithms use only the questions that both potential matches decide to answer, and the match questions McKinlay had chosen—more or less at random—had proven unpopular. When he scrolled through his matches, fewer than 100 women would appear above the 90 percent compatibility mark. And that was in a city containing some 2 million women (approximately 80,000 of them on OkCupid). On a site where compatibility equals visibility, he was practically a ghost.

He realized he’d have to boost that number. If, through statistical sampling, McKinlay could ascertain which questions mattered to the kind of women he liked, he could construct a new profile that honestly answered those questions and ignored the rest. He could match every woman in LA who might be right for him, and none that weren’t.
 

I expected the article to be about how his complex hacking led quickly to the perfect woman, but what's surprising (or depressing, depending on how much faith you put in market efficiency) is how much work he had to go through even after his complex data mining surfaced a candidate list.

Far easier, I suppose, if you are able to nail the three rules mentioned in the old Tom Brady sexual harassment sketch on Saturday Night Live:

1. Be Handsome.
2. Be Attractive.
3. Don't Be Unattractive.

I don't envy the best man at McKinlay's wedding, having to recount the story of how the couple met.

The sofalarity

The Oji-Cree have been in contact with European settlers for centuries, but it was only in the nineteen-sixties, when trucks began making the trip north, that newer technologies like the internal combustion engine and electricity really began to reach the area. The Oji-Cree eagerly embraced these new tools. In our lingo, we might say that they went through a rapid evolution, advancing through hundreds of years of technology in just a few decades.

The good news is that, nowadays, the Oji-Cree no longer face the threat of winter starvation, which regularly killed people in earlier times. They can more easily import and store the food they need, and they enjoy pleasures like sweets and alcohol. Life has become more comfortable. The constant labor of canoeing or snowshoeing has been eliminated by outboard engines and snowmobiles. Television made it north in the nineteen-eighties, and it has proved enormously popular.

But, in the main, the Oji-Cree story is not a happy one. Since the arrival of new technologies, the population has suffered a massive increase in morbid obesity, heart disease, and Type 2 diabetes. Social problems are rampant: idleness, alcoholism, drug addiction, and suicide have reached some of the highest levels on earth. Diabetes, in particular, has become so common (affecting forty per cent of the population) that researchers think that many children, after exposure in the womb, are born with an increased predisposition to the disease. Childhood obesity is widespread, and ten-year-olds sometimes appear middle-aged. Recently, the Chief of a small Oji-Cree community estimated that half of his adult population was addicted to OxyContin or other painkillers.
 

From Tim Wu over at The New Yorker. What is the sofalarity?

Technological evolution has a different motive force. It is self-evolution, and it is therefore driven by what we want as opposed to what is adaptive. In a market economy, it is even more complex: for most of us, our technological identities are determined by what companies decide to sell based on what they believe we, as consumers, will pay for. As a species, we often aren’t much different from the Oji-Cree. Comfort-seeking missiles, we spend the most to minimize pain and maximize pleasure. When it comes to technologies, we mainly want to make things easy. Not to be bored. Oh, and maybe to look a bit younger.

Our will-to-comfort, combined with our technological powers, creates a stark possibility. If we’re not careful, our technological evolution will take us toward not a singularity but a sofalarity. That’s a future defined not by an evolution toward superintelligence but by the absence of discomforts.
 

Ah yes, it's the Wall-E scenario. If we are not careful, technology may lead us to a corrupted Maslow's hierarchy of needs in which the top rung is not self-actualization but the absence of discomfort.

Maybe we will follow a trail of in-app purchases there. Someone who has finished Candy Crush Saga, let me know if, when you complete the game, a video plays showing the developer laughing at you and shouting, “Suckerrrrrrrr!”

Why don't people have more sex?

An older but still amusing post from Marginal Revolution.

3. Freud was right and we are all repressed.  The will is not unitary and the utility-maximizing part is not always in control.

4. There has been a market failure, but the Internet is remedying it.  People are having more sex and this will only go up.

5. Sex stops being fun when you do it to close a gap between your marginal utilities.  It requires spontaneity or some other quality inconsistent with the classical model of the consumer and the equation of marginal rates of substitution.

Where physical and digital spaces meet

For his dissertation at the University of Toronto, Hampton studied an extraordinary early experiment in wired living. In the mid-1990s, a consortium that included IBM and Apple helped raise more than $100 million to turn a new suburban development in Newmarket, Ontario, a Toronto suburb, into the neighborhood of the future. As houses went up, more than half of them got high-speed Internet (this in the age of dial-up), advanced browser software for their computers, a tool for videoconferencing between houses and a Napster-like tool for music sharing. He treated the other homes as a control group. From October 1997 through August 1999, Hampton lived in a basement apartment in the new development, observing and interviewing his neighbors.

Hampton found that, rather than isolating people, technology made them more connected. “It turns out the wired folk — they recognized like three times as many of their neighbors when asked,” Hampton said. Not only that, he said, they spoke with neighbors on the phone five times as often and attended more community events. Altogether, they were much more successful at addressing local problems, like speeding cars and a small spate of burglaries. They also used their Listserv to coordinate offline events, even sign-ups for a bowling league. Hampton was one of the first scholars to marshal evidence that the web might make people less atomized rather than more. Not only were people not opting out of bowling leagues — Robert Putnam’s famous metric for community engagement — for more screen time; they were also using their computers to opt in.
 

Lots more here, all interesting. Of course, since that time the iPhone and smartphones in general descended on the world and pushed us even deeper across the membrane separating the physical world from the digital one, but it's good to see actual quantitative research into the effects of technology. The reflexive reaction is generally negative.

Based on visits to parks and plazas in New York, Los Angeles, Seattle, Boston, Minneapolis, Montreal and Venice, Whyte and his acolytes formulated conclusions that were, for their time, counterintuitive. For example, he discovered that city people don’t actually like wide-open, uncluttered spaces. Despite the Modernist assumption that what harried urban people need are oases of nature in the city, if you bother to watch people, you see that they tend to prefer narrow streets, hustle and bustle, crowdedness. Build a high-rise with an acre of empty plaza around it, and the plaza may seem desolate, even dangerous. People will avoid it. If you want people to linger, he wrote, give them seating — but not just benches, which make it impossible for people to face one another. Movable chairs can be better. Also: Never cordon off a fountain. “It’s not right to put water before people and then keep them away from it,” Whyte wrote in his 1980 book, “The Social Life of Small Urban Spaces.” People want to splash, dip their toes, throw coins. He believed that dense greenery can make places feel less safe, that people find the fishbowl effect of sunken plazas disconcerting and, presciently, that food trucks draw crowds. Whyte’s insights were incorporated into 1975 revisions of New York’s zoning code, and the Bryant Park Corporation — credited with turning around the once-squalid park — bases its work on many of his principles.

 

The Social Life of Small Urban Spaces collects more of these fascinating observations on urban design from William Whyte. I browsed my friend's copy years ago and found a lot that was counter-intuitive to this armchair urban planner.

First off, mobile-phone use, which Hampton defined to include texting and using apps, was much lower than he expected. On the steps of the Met, only 3 percent of adults captured in all the samples were on their phones. It was highest at the northwest corner of Bryant Park, where the figure was 10 percent. More important, according to Hampton, was the fact that mobile-phone users tended to be alone, not in groups. People on the phone were not ignoring lunch partners or interrupting strolls with their lovers; rather, phone use seemed to be a way to pass the time while waiting to meet up with someone, or unwinding during a solo lunch break. Of course, there’s still the psychic toll, which we all know, of feeling tethered to your phone — even while relaxing at the park. But that’s a personal cost. From what Hampton could tell, the phones weren’t nearly as hard on our relationships as many suspect.

When I met Hampton, he proved this point by gesturing around us, at our fellow diners at the Bryant Park Grill, where we were eating on a beautiful summer day, and at the hundreds of others beyond us in the park, enjoying the sun at tables, in chairs and on the lawn beyond us. “In the busiest public spaces, where there are a lot of groups, like this kind of public space, it’s like 3 percent,” he said. “Three percent. I can’t even see someone on a cellphone right now, but yet how many times have you seen a story that says, ‘People on cellphones in public spaces is rude, it’s creating all sorts of problems, people are walking into traffic.’ I mean, we really have a strong sense that it’s everywhere.”

Hampton’s project offers an explanation for that misperception. It turns out that people like hanging out in public more than they used to, and those who most like hanging out are people using their phones. On the steps of the Met, “loiterers” — those present in at least two consecutive film samples, inhabiting the same area for 15 seconds or more — constituted 7 percent of the total (that is to say, the other 93 percent were just passing through). That was a 57 percent increase from 30 years earlier. And those using mobile phones there were five times as likely to “loiter” as other people. In other words, not that many people are talking, or reading, texting or playing Candy Crush on the phone, but those who do stick around longer.
 

It's commonly accepted that not only are smartphones are stealing people's attention from the real world, it's not socially healthy. Our attention spans are getting shorter, we are not paying attention to other people, our relationships are more shallow and transient when pulled through the narrow digital pipes of social networking services.

More omnipresent internet connectivity, smartphones, and the proliferation of Internet services that are never-ending firehoses of information of variable quality have turned the world into a giant information Skinner box, and I'm as guilty as anyone of succumbing to the temptation. Most of us spend more of our lives now retreating from the physical world into a world of information.

Is that inherently bad? Conventional wisdom has been a harsh judge, but for now I'm withholding judgment. I don't doubt there are some truly perverse downsides at the extremes on sociability, physical health, and perhaps even mental focus, but I'd love more in-depth discussion of the positives. For example, television, regarded in decades past as intellectually corrosive, has been proven to have significant positive social effects in countries where it becomes widespread.

Did the free market make us irrational?

A well-known example of irrational decision-making people's tendency to overvalue the things they own (I would pay $1 for a coffee mug but will demand $5 for an identical coffee mug that happens to be mine). This bias of "the mind" is called the "endowment effect" and is often assumed to be universal (and therefore explained as the work of evolution). But in this paper Coren Apicella, Eduardo Azevedo, James Fowler, and Nicholas A. Christakis found that some people and some minds don't have this bias at all. Rather than being built-in to human nature, they write, the endowment effect may be a habit of mind that people learn in market-oriented societies. If that's true, it means that (for this trait at least) the hunter-gatherers described in the research were more rational before they were exposed to modern capitalism.
 

Fascinating.

In any case, the other important lesson of this paper is that, as ever, theories about "the mind" shouldn't be based on tests run only on minds that are Western, Educated, Industrial, Rich and Democratic, or WEIRD. And that, more generally, it's always important to check one's assumptions about what is innate and universal in psychology. "Whenever a pattern of human behavior is widespread, there is reason to suspect that it might have something to do with our evolutionary history," a pair of biologists recently noted. True enough, but sometimes patterns of human behavior aren't as widespread as we want to believe, in our eagerness to spin a theory. (People have, in fact, tried to find a reason why the endowment effect must have evolved to be a part of every human psyche—for example, here.) The attractiveness of such theories should make everyone a little cautious about the generalizations on which they rest.