How Do Superforecasters Predict the Future? Very, Very Carefully.

Famous pundits virtually never make falsifiable forecasts.

Philip Tetlock: We know virtually nothing about the forecasting track records of famous pundits because famous pundits virtually never make falsifiable forecasts. They say something might happen or could happen or may very well happen, but when I say something could happen, that doesn't mean a lot. I mean it could be that it could happen that we're all going to be vaporized by an asteroid in the next 24 hours, or it could be that the sun will rise tomorrow. "It could" subsumes an enormous range of possible probabilities. So if I say something could happen and it does happen, I can say to my readers, "Well, you know, I told you it could." And if I say something could happen and it doesn't happen, I can come back to my readers and say, "I just said it could." One of the interesting things about superforecasters is how opportunistic they are. Superforecasters think quite strategically about when it makes sense to invest effort in thinking. So if you wanted to predict the outcome of the presidential election in early October 2015, 2016 presidential election, how would you go about it if you were a superforecaster? A superforecaster wouldn't look carefully at the presidential debates, look into the eyes of the candidates and see which one looks more presidential. At some point in the process a superforecaster might do that, but a superforecaster would tend to start with more the outside view and gradually work in rather than start at the inside and work out. So they would ask very general questions initially like: Let's look at elections, all presidential elections since World War II; how likely is a Democrat or a Republican to win? Or they might say after one party has held the presidency for two terms, how likely is there to be a transition? Or they might say if economic growth is less than 2 percent three quarters before the presidential election, does that bode ill for the party that controls the presidency?

So they would start off with these more general estimates, these more outside view estimates and then they would gradually adjust in response to estimates about popularity polls of candidates, which are notoriously volatile at this stage of the process. But they would take them into account, but they would discount them considerably because they are so volatile and then they would adjust incrementally. So the best forecasters, I think, over the course of 2015, have somewhat lowered their probability estimate of Hillary Clinton being the next president of the United States. I think they started off significantly above 50 percent and there's been some significant hemorrhaging of those estimates, but nothing all that dramatic. I mean the interesting thing about superforecasters is they're very patient and they make granular belief updates. So they don't suddenly say, "Oh my God, the latest email scandal," or, "Is Biden going to come into the race?" or this or that, there are these little clues and it's not that they ignore them but they tend to respond very incrementally to them. So it may be the probability of Hillary Clinton becoming the next president of the United States moves from the 45 percent to 43 percent in response to an inspector general report in the State Department, that sort of thing. The best forecasters tend not to make it on to television. They’re not very attractive to the TV producers because they’re much more likely to say, "On the one hand; on the other hand." And then now we have to strike some sort of integrated balance and that apparently doesn't make very good television.

"Famous pundits virtually never make falsifiable forecasts." To abide by that simple axiom is to ensure oneself a terrific mode of self-preservation, whether you're Nate Silver or Jim Cramer or any of the other famous personalities making a living off of the science of prediction.

The science of prediction is of particular interest to University of Pennsylvania professor Philip Tetlock, a political scientist who has written a new book on the subject: Superforecasting: The Art and Science of Prediction


In this video, Tetlock runs through the thought processes of professional media prediction-makers whose livelihoods depend not on being right, but on being right enough.

NASA astronomer Michelle Thaller on ​the multiple dimensions of space and human sexuality

Science and the squishiness of the human mind. The joys of wearing whatever the hell you want, and so much more.

Flickr / 13winds
Think Again Podcasts
  • Why can't we have a human-sized cat tree?
  • What would happen if you got a spoonful of a neutron star?
  • Why do we insist on dividing our wonderfully complex selves into boring little boxes
Keep reading Show less

How to split the USA into two countries: Red and Blue

Progressive America would be half as big, but twice as populated as its conservative twin.

Image: Dicken Schrader
Strange Maps
  • America's two political tribes have consolidated into 'red' and 'blue' nations, with seemingly irreconcilable differences.
  • Perhaps the best way to stop the infighting is to go for a divorce and give the two nations a country each
  • Based on the UN's partition plan for Israel/Palestine, this proposal provides territorial contiguity and sea access to both 'red' and 'blue' America
Keep reading Show less

Increased air travel may decrease the chances of a global pandemic

The closer together we get, the argument goes, the healthier we'll be.

Surprising Science
  • The more exposed we are to each other, the less surprising a pathogen will be to our bodies.
  • Terrorism, high blood pressure, and staffing issues threaten to derail progress.
  • Pursuing global health has to be an active choice.
Keep reading Show less