Can Twitter predict the Stock Market? Twits, Butter and the Super Bowl Effect

As we more or less know, the sheer randomness of the world makes predicting stockmarket movements not so much a fool's game as Russian Roulette. If you believe you can outwit the markets on a day to day basis it's only a matter of time before the hammer falls on a firing cap. 

Despite this people keep trying, because it's a basic human urge to try to make sense out of the nonsensical. The ancient Chinese believed a solar eclipse was caused by a celestial dragon munching up the Sun. At the time that counted as advanced thinking: indeed it still does in some parts of the world, but generally it's a lot hard these days to believe that people think stockmarket movements can be predicted by the random and trivial ramblings of posters on Twitter.

Can't be right, can it?

Moody Twits Drive the Markets

In Twitter Mood Predicts The Stock Market the researchers find that Twitter mood is 87.6% successful in predicting the daily movements of the Dow Jones Industrial. The researchers suggest that this is an outcome of behavioral finance: the idea being that "financial decisions are driven by emotion and mood".

Which is true, to some extent, but it rather beggars belief that almost 90% of daily stockmarket movements are attributable to this, even more so that the world's stockmarket movers intersperse their frantic trading to bother explaining how they're feeling. Far be it from this less than august journal to argue in favour of the Efficient Markets Hypothesis, but most stock market movements aren't that irrational most of the time.

The study employs lots of whizzy technology including a "Self-Organizing Fuzzy Neural Network" which sounds like a handy gizmo to have around the office, but is basically a program that has learned to make predictions based on past data – in this case either the last three days of market movements or the same data with various bits of mood related data cleverly extracted from Twitter.

Bangladeshi Butter

To be honest it's a stunning result but its very accuracy raises questions about whether it's actually measuring some real correlation with stockmarkets or whether it's simply a repeat of the case of the Bangladeshi Butter. Because, oddly enough, Bangladeshi butter production figures have also been shown to be a darn fine predictor of stockmarket movements.

In one of the classic papers that all first year economic graduates should be made to learn by heart David Leinweber showed exactly how easy it is to find some way of predicting market movements – in hindsight. However, as Stupid Data Miner Tricks also shows, these models have about as much future predictive ability as a government finance minister in election year.

The Super Bowl Effect

To give another example, one of the more remarkable forecasters of future market movements was the sportswriter Leonard Koppett who successfully predicted the direction of the S&P 500 for many, many years. His original Sporting News article from 1978 shows an eleven year streak, and ended up being correct for 18 out of 19 years.

Impressive, you might think, but your regard for him as a stock market pundit will probably be slightly reduced when you discover that his predictive algorithm was based on whether the winner of the Superbowl came from the AFC or the NFC. The general point, of course, is that you don't need to invoke complex explanations of behavioral economics when a rather more straightforward explanation is the randomness of the world.

(In)credible Correlations

Which is also what Leinweber set out to show. He too took the S&P 500 and went looking for something – anything – correlated with it. He found:

"Butter production in Bangladesh. Yes, there it is. A simple dairy product that explains 75% of the variation of the S&P500 over 10 years"

By adding in US butter production, US cheese production and the sheep population in those two countries he was able to improve his accuracy to 99%. The point being, of course, that no one in their right mind would believe that this particular set of variables is in any way whatsoever causing the movements of a US stockmarket. However, as he goes on to point out, it's rather too easy to take exactly the same approach and make it sound like it makes sense:

"If someone showed up in your office with a model relating stock prices to interest rates, GDP, trade, housing starts and the like, it might have statistics that looked as good as this nonsense, and it might make as much sense, even though it sounded much more plausible".

Patrick Burns has performed an out of sample analysis of the aforementioned Super Bowl Effect, which is interesting in its own right, but ends with a neat summary of how psychological expectations condition our belief, or otherwise, in the predictive capability of economic models:

"People are perfectly willing to believe that economic variables may have predictability for stock markets, but sporting variables are given close to zero credibility. It would take overwhelming evidence to get people to invest based on the outcome of a ball game. To put it in statistical jargon, investors act like Bayesians - results from data are tempered by prior beliefs".

Problem Models

In fact it's simple to construct a model out of historical data. Take two arbitrary data points and connect them via a straight line, make the y axis time and the x axis stockmarket returns. It's trivially easy to now predict next year's return, and highly improbable that the prediction will be be correct. However, just adding more data points doesn't make the model's predictive capability any better. Leinweber starts out with ten data points:

"We have 10 points on the S&P annual series from 1983 to 1992 … and it hits every annual close exactly. We've got 100% in-sample accuracy with only one variable … What closing value the S&P did this method predict for the end of 1993? Minus 63,311. Fortunately for the global economy , it actually turned out to be positive, +445. We seem to have a problem with our model's out of sample performance".

The basic problem is something we've looked at before: "the market has only one past". Odd though this may be it actually means we only have one sample to base our models on. What we actually need is to be able to take the data from an infinite number of universes. However, we can't do that yet so we have to be suitably suspicious of anyone waving around a plausible and unexploded new correlation.

Death by Data Mining

There is so much data out there and so much computing power capable of crunching it, looking for correlations, that there are bound to be very good fits between some data sets. That doesn't tell us anything about causality but, of course, the more plausibly related the data sets are the more likely we are to be convinced. Yet if behavioural economics tells us anything, it's that it's highly unlikely that complex systems including human beings can be reduced to simple correlations. In the case of our Twitter research, it's a nice conceit that market movements can be reduced to the random babblings of twitchy tweeters through some kind of wisdom of the crowds behavioural effect. It even sounds sort of plausible. However, as Leinweber points out, in nice friendly large letters:

If it seems to be too good to be true it probably is.

Damn right. Read the paper.