Cognitive Science · Human Thinking · The Human Mind and Brain

“Super-forecasting: The Art and Science of Prediction” by Philip Tetlock

Tetlock says he is an skeptical optimist:  He says chaos theory makes him a pessimist.  But still, some things are predictable.  We all use mundane predictability all the time.  For example traffic is bound to be bad at rush hour.  

Some people are much better at forecasting than others.  It is not what they are: smarter, more knowledgeable, better at math.  It is what they do.  Basically, good forecasters tend to test their predictions and measure how right or wrong they actually are and then try to refine their predicting.  They apply scientific method to testing.

There are some hard limits to forecasting.  Some things are just not predictable.  For example how the fruit vendor in Tunisia triggered the mideast spring.  

Edward Lorenz in 1972 wrote a paper about the sensitivity of initial conditions in which he asked the question about whether a butterfly flapping its wings in Brazil can cause a tornado in Texas. This was the beginning of chaos theory and the realization that in complex systems predictability is probably impossible. A minor change in initial data in a complex system leads to wildly different results a few iterations into the future.  This suggests that there may be hard limits on predictability, especially in complex systems.

Nonlinear systems is another term for complex systems or chaos theory.

In a deterministic clockwork world it is possible to predict the future accurately. Chaos theory ended that certainty. The certainty of prediction came out of Newton’s three laws and was codified by Laplace who said absolute prediction was possible as long as you knew with certainty the velocity and position of every particle in the universe.

There are no certainties in life but there are probabilities.

Without measuring, in other words checking whether forecasts are actually correct, there is no improvement in forecasting.  The process is: forecast, measure, revise.  This is the only way to make the world better.

“Behavioral science is the systematic analysis and investigation of human and animal behaviour through controlled and naturalistic observation and disciplined scientific experimentation. It attempts to accomplish legitimate, objective conclusions through rigorous formulations and observation.”  wiki.  This is the process that makes forecasting work much better.  

The attributive of bias is one of the major biases identified in the book thinking fast and thinking slow. The attributive bias is what happens when people need to find  reasons why something happened or is going to happen. Human beings need to  find patterns, they love patterns, they need to find reasons why things are going to happen. They don’t want to say well I just don’t know why this happened. So they invent reasons why things happen that make themselves look like geniuses and that they had it figured out all slong.  This seems to be a major reason why things go wrong. All of us do this all of the time. We want to make what happens reasonable and rational and that we know the reasons. So we find all kinds of little pieces of evidence that seem to go together and fill out the pattern even if thede reason are mostly nuts, which is what happens most of the time.

You can’t test a forecast unless there is a definite time line. Also terms have to be defined so that you know exactly what the forecast is saying.  Also probability has to be clearly spelled out. You have to use terms like very possible, slight possibility, etc.  And actually it is much better if  possibilities can be stated numerically and quite explicitly. For example “the odds are 80/20 that X will happen, this is very specific.  Weather forecasters always express things in terms of percentage. For example, tomorrow there is a 30% chance of rain.


The article is still in progress.  It will be finished soon.

Foggy fall day in the Maine Blueberry barrens.  Picture by Hanselmann Photography. 

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s