The 2012 Election: A Big Win for Big Data

This was indeed a choice election, if you consider the choice between consuming entertainment journalism or data-based journalism. Entertainment is fun, and math is hard. Math won. 

"We begin with the three words everyone writing about the election must say: Nobody knows anything," wrote Peggy Noonan the day before the 2012 election. 

One man, however, posted a seemingly audacious election forecast that gave Barack Obama over a 90 percent chance of victory in the Electoral College. He is the statistician and New York Times blogger Nate Silver. Silver has been a bit of a lightning rod, as one would expect in a hotly contested election. 

Some of his critics objected that he had a horse in the race -- he is an avowed Obama supporter. Other critics objected that he didn't have a horse in the race -- he wasn't putting his money where his mouth was. So Silver did offer up a $1,000 wager with the MSNBC host Joe Scarborough. 

However, Silver had a lot more riding on this election than the money. His integrity was on the line, and along with it the integrity of the fast-evolving field of sabermetric analysis. 

But Silver got it right, once again, and his 538 model, a statistical assessment of national and statewide polls that also factors other indicators like economic data and state-by-state demographic fundamentals, has become the closest thing to a crystal ball that there is in politics today.

What's the Big Idea?

This is what Silver's well-deserved victory lap should really mean for all of us. While some polls need to update their methodologies, such as Gallup, which doesn't include cell phones in its surveys, data analysis is growing ever smarter. And since we're talking about the science of forecasting, let's propose the idea, as many innovators in this field have, that new methods of data analysis will continue to emerge, greatly improving our ability to predict the future. 

The ramifications of Silver's successful forecasting, of course, goes well beyond politics. After all, Silver, the author of The Signal and the Noise: Why So Many Predictions Fail-but Some Don'tcut his teeth in sabermetrics -- the same stuff Billy Beane, of Moneyball fame, used to make the small market Oakland A's competitive with mega-market teams like The New York Yankees.

Big Data analysis has implications for forecasting hurricanes and shifts in the economy. In fact, Big Think blogger Dominic Basulto has argued that the use of Big Data is "the key to unlocking growth for smaller companies throughout the U.S. economy."

What's the Significance?

Just like in baseball and politics, there are winners and losers in a data-driven world. The losers in baseball, for instance, are the over-rated prospects who will never be drafted because data analysis has a way of finding them out early on in their careers. In politics, I would like to believe the biggest loser will be the horse race pundit, the guy who spins the polls to reinforce one side's belief that it is winning when it's actually losing. Sometimes this is done for partisan reasons, in the hope of creating "momentum," and sometimes it is done to create a more compelling media narrative. This was indeed a choice election, and the choice was between following entertainment journalism or data-based journalism. As Andrew Beaujon pointed out, entertainment is fun, and math is hard. Well, math won. 

So what makes Nate Silver more numerate than the rest of us? In one sense, not a lot. As Silver himself says, all he did was take an average of polls and count to 270. But there's another skill that we need to develop to master data. We need to kick the habit of killing the messenger. As Silver himself has pointed out, Republicans loved him and Democrats hated him in 2010, when his forecast indicated a GOP takeover of the House of Representatives. Silver's admirers and haters changed seats at the dinner table this election cycle, when Silver accurately predicted a Democratic win. This is obviously silly business. We need to have a mature appreciation for data, even if it doesn't tell us what we want to hear. That may be a tall order for a Presidential election, but will have much greater ramifications for who wins and who loses in the ultra-competitive business landscape of the 21st century.  

Follow Daniel Honan on Twitter @Daniel Honan

Stand up against religious discrimination – even if it’s not your religion

As religious diversity increases in the United States, we must learn to channel religious identity into interfaith cooperation.

Sponsored by Charles Koch Foundation
  • Religious diversity is the norm in American life, and that diversity is only increasing, says Eboo Patel.
  • Using the most painful moment of his life as a lesson, Eboo Patel explains why it's crucial to be positive and proactive about engaging religious identity towards interfaith cooperation.
  • The opinions expressed in this video do not necessarily reflect the views of the Charles Koch Foundation, which encourages the expression of diverse viewpoints within a culture of civil discourse and mutual respect.
Keep reading Show less

The problem with problem solving? It isn’t ridiculous enough.

Ask very silly questions to spur very serious innovation.

  • To get really innovative solutions to complex problems, you need to abandon logic, says Dan Seewald.
  • Asking provocative and ridiculous 'what if?' questions pushes us down lateral paths of thinking versus the vertical or logical path. The latter approach is practical but it doesn't break new ground.
  • Breaking with tradition through lateral thinking allows us to solve really serious problems, from climate change to political turmoil. Or, as Dan Seewald explains, it could just help you solve all your laundry headaches.
Keep reading Show less

10 new things we’ve learned about death

If you don't want to know anything about your death, consider this your spoiler warning.

Culture & Religion
  • For centuries cultures have personified death to give this terrifying mystery a familiar face.
  • Modern science has demystified death by divulging its biological processes, yet many questions remain.
  • Studying death is not meant to be a morbid reminder of a cruel fate, but a way to improve the lives of the living.
Keep reading Show less

Where the evidence of fake news is really hiding

When it comes to sniffing out whether a source is credible or not, even journalists can sometimes take the wrong approach.

Sponsored by Charles Koch Foundation
  • We all think that we're competent consumers of news media, but the research shows that even journalists struggle with identifying fact from fiction.
  • When judging whether a piece of media is true or not, most of us focus too much on the source itself. Knowledge has a context, and it's important to look at that context when trying to validate a source.
  • The opinions expressed in this video do not necessarily reflect the views of the Charles Koch Foundation, which encourages the expression of diverse viewpoints within a culture of civil discourse and mutual respect.
Keep reading Show less