Barack Obama was not the only winner in the 2012 presidential election. Nate Silver, now founder and editor in chief of Five Thirty Eight, and other stats-y election forecasters basked in the praise that came when the returns matched their predictions.

But part of the praise was overstated. At the very end, Silver’s models essentially called Florida a toss-up, with the probability of an Obama win going just a few tenths of a percentage point above 50%. But because his model gave Obama the slightest of edges in Florida, his forecast in most of the media essentially became a predicted Obama win there. In addition to accurately forecasting the national popular vote, Silver then received credit for predicting all fifty states correctly.

I am all in favor of stats winning, but the flip side of this is the problem. If Obama had not won Florida, Silver’s prediction―which, like that of other forecasters such as Sam Wang of the Princeton Election Consortium, was excellent―would have been no less good. ^[1]This is a column about football, but you might want to check out some of the stuff through that link on the differences between Silver and Wang on the upcoming midterm elections. They both know way … Continue reading And if stats folks bask too much in the glow when everything comes up on the side where the probabilities leaned, what happens the next time when people see a 25% event happening and say that it invalidates the model? ^[2]Of course, maybe Football Outsiders has already run into that with the 2007 Super Bowl prediction. Perhaps sports people are ahead of politics on this stuff.

Lots of people have made this point before — heck, Silver wrote about this in his launch post at the new 538 — but it is really useful to think carefully about the uncertainty in our predictions. Neil has done that with his graphs depicting the distribution of team win totals at 538, and Chase did so in this post last Saturday. Football Outsiders does this in its Almanac every year, with probabilities on different ranges of win totals. [continue reading…]