By Nate Silver
Several years ago when we were preparing for Highlands Forum 41 on “The Frontier of Prediction”, a friend of Highlands who was perhaps the most knowledgable person we know of on all things baseball directed us to the website of an interesting young man who had insights on many new statistical measurements for baseball productivity. The young man was Nate Silver, and at the age of 24 he had developed a statistical measurement that accurately forecast performance of comparable players. It became a valuable tool, not just for “seamheads” (baseball fantasy league denizens who use Sabermetrics as tools for making determinations—see the 2003 Highlands Forum reading list entry on Moneyball by Michael Lewis, which was recommended that year by Johns Hopkins University President William Brody), but for baseball executives as well who make the real world decisions with ownership budgets. In 2008 Silver examined political polling and forecasting and thought it specious at best, so he decided to bring his methodologies to electoral predictions with his blog FiveThirtyEight. He made his name in the broader world (away from baseball) of politics and media with again, stunning accuracy. What is interesting for us in Highlands is Silver’s appreciation for the rigor applied to understanding prediction by Phil Tetlock, our featured speaker at Highlands 41. Silver, in his very influential new book, The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t, says: “Tetlock’s conclusion was damning. The experts in his survey (Expert Political Judgment)—regardless of their occupation, experience, or subfield—had done barely any better than random chance, and they had done worse than even rudimentary statistical methods at predicting future political events”. Silver’s story is cautionary: he tells us early on, “We face danger whenever information growth outpaces our understanding of how to process it…Data-driven predictions can succeed—and they can fail. It is when we deny our role in the process that the odds of failure raise. Before we demand more of our data, we need to demand more of ourselves”. Perhaps that is why so many were taken in by misleading presidential polling results in 2012, with many pundits citing those that agreed with their choice of candidate (or worse, when pollsters asked questions or structured polls that were bound to give biased results and candidates, party loyalists, and voters believed them). Many were shockingly abysmal, and downright wrong. How did Silver do? Unlike most pundits, he called all fifty states correctly and was right in the margin on the popular vote. But Silver takes success in stride with a final warning in The Signal and the Noise: “May we arise from the ashes of these beaten but not bowed (unpredicted disasters of the past twelve years), a little more modest about our forecasting abilities, and a little less likely to repeat our mistakes”. Silver’s book is likely to be one of the more influential of this or other years as it is flying off the shelves—for those who read it, be sure to remember to think about the certitude of bold predictions, even by, and perhaps, especially by “experts” in any endeavor. Read this book (and thanks to Jim for the tip!).