King Kaufman mentioned this article before, and now so does the NYTimes sports page: "Underestimating the Fog" by Bill James (PDF). James writes that if the Baseball Research Journal were a scientific journal, he would have titled the article "The Problem of Distinguishing Between Transient and Persistent Phenomena When Dealing with Variables from a Statistically Unstable Platform".

James notes that many widely held sabermetric tenets, including the non-existence of clutch hitting, may be based on a faulty statistical method. Until now, sabermetricians have believed that if a player does own a skill, like clutch hitting ability, it should persist from year to year. Fluctuations in a player's hitting performance in clutch situations from year to year has been taken to mean that clutch hitting is a myth. Some other ideas based on this statistical method (all of which should be familiar to any reader of Bill James Baseball Abstracts, Baseball Prospectus, Rob Neyer, or any of their countless web acolytes) include the following:

Pitchers can't control wins and losses. They can only prevent the other team from scoring runs.

Team performance in one-run games is largely luck.

Catchers don't affect their pitchers' ERAs

Pitchers can't control batting average of balls put in play (recent studies have qualified this to say that pitchers largely cannot control this).

Batters do not go on hot or cold streaks.

James does not argue that any of these are false, but he is concerned that the proofs are faulty.

"We ran astray because we have been assuming that random data is proof of nothingness, when in reality random data proves nothing...random data proves nothing--and it cannot be used as proof of nothingness.

Why? Because whenever you do a study, if your study completely fails, you will get random data. Therefore, when you get random data, all you may conclude is that your study has failed."

It's a fantastic article. Sabermetric thinking hasn't made any revolutionary leaps relative to itself in a long time. That it is taking hold in more baseball organizations now may make it seem fresh, but many of the ideas have been around for some time (of course, teams may have come upon some strategies that they have kept secret; I'm referring only to publicly available ideas). James's last Baseball Abstract came out in 2001. He doesn't seem to write for the public much anymore now that he works for the Red Sox (though recently he did conduct this interview with Sons of Sam Horn). I miss his contrarian voice--he's a bit of a curmudgeon, and whenever I catch the random episode of House I feel that Dr. Gregory House (Hugh Laurie) is a medical version of Bill James.

James still believes that hot streaks do not exist. This is one phenomenon I believe in without any data as proof. But in my own experience playing sports, I've gone through productive runs that I'd term hot streaks, during times when I've felt "in the zone." In baseball, it's the sense that all the pitches are slower and the ball larger than normal, easier to pick up with the eye. In basketball, it's the feeling that every jump shot will go in, that the basket is larger than normal. Personal experience and sensation is notoriously unreliable as a way to explain the world, so my belief in the existence of hot streaks is an article of faith. No one has ever been able to explain the phenomenon physiologically, but players refer to it again and again in different sports. If it is truly a transient (non-persistent) and random occurrence, these hot streaks, then they would indeed be very difficult to detect.

On the other hand, I can muster no amount of faith in the Cubs this season, and that was even before Nomar Garciaparra and Todd Walker went down with injuries. Not enough hitters, injury risks galore in the pitching staff, and a manager with questionable tactical ability. No underestimating that fog; it's hanging over Wrigley, and hanging over my head. 90% chance of showers.