Lessons In Investing From Science

8th Feb '16 by timarr

6 comments 9228 reads

Confirm Ye Not

Here's what ought to be a really boring idea - we need scientists in general and psychologists and economists in particular to stop hypothesising after results are known (HARKing, geddit?). Instead they need to state what they're looking for before they conduct their experiments because otherwise they cherrypick the results they find to confirm hypotheses they never previously had.

The underlying problem is our old foe, confirmation bias. And the solution for scientists and social scientists alike is known as pre-registration. It would be no bad thing for investors to demand a similar process for fund managers and financial experts. Or, for that matter, to apply some of the ideas to their own investing strategies.

No No Negatives

It's been known for years that a lot of scientific research isn't very reliable. There are numerous problems, chief amongst them being the non-publication of negative results: an issue known as publication bias. There's no kudos in showing that your hypotheses were wrong, so researchers and corporations tend to bury the data, but it's still valuable information that should be shared: scientists see further by standing on the shoulders of others, we shouldn't be encouraging them to shrug them off because they've got bored.

Repeat, Again

Once we move to the social sciences then the problems are even worse. Human beings are terrible things to experiment on, being inclined to change their minds, develop opinions about the experiments and to second-guess what the researchers would like them to do, just to be nice.

All too many experiments in the social sciences turn out to be flawed because of social or situational factors that didn't seem important at the time. Given this you'd think that repeating experiments to make sure the results held would be even more important for psychologists than it is for researchers in the hard sciences. Well, guess again.

According to research by Matthew Makel, Jonathan Plucker and Boyd Hegarty only a little over 1% of psychology studies have ever been replicated. Everything else is simply a matter of faith in the integrity and lack of bias of the original researchers. Which is not science: in the words of John Tukey, quoted at the head of their paper:

"Confirmation comes from repetition. Any attempt to avoid this statement leads to failure and more probably to destruction."

Pre-Register

The best…

Unlock with Email

Unlock with Google

Already have an account?
Login here

rhomboid1

8th Feb '16

Great post Timarr, over the years I've made a number of howlers through not understanding my own bias , more specifically I view current events through the prism of my experience and end up surprised when history fails to be an infallible guide to the future. Basically numpty mistakes..

Nick Ray

The problem is that the performance of the market is anti-inductive as Taleb calls it. That is, you cannot deduce how it works using induction. Because it contains intelligent agents it is able to observe attempts to understand it and react to them.

In particular, if you did ever find a way to beat the market (average) 100% of the time, the one thing you could not afford to do is publish it!

For me, a useful model to understand stock movements is based on geometric brownian motion. You have to try to identify the 'drift' component in the face of a much larger volatility component. The volatility component will integrate to zero over a long enough time frame (by definition) and the drift component gradually asserts itself. Of course a complicating factor is that the 'drift' for any given stock can change over time.

Although you can often view the drift quite easily by eye using a log-price scale and a suitable time frame (a year or longer), ultimately the drift is linked to fundamental factors. While those remain strong a stock will perform well over "long enough" time periods.

By contrast, over short time periods the market attempts to find a price for each stock which means that the expected return for any stock is the same (so good stocks become more "expensive" than riskier stocks). Trying to outwit this mechanism means exploiting so-called inefficiencies in the market. This is a zero-sum game which one strongly suspects can only be consistently won with information not generally available to all market participants.

Mark Carter

I once tried putting some shares through a fourier transform. There were some problems with my methodology, though. I was trying to see if there were any cycles. Nothing leapt out at me particularly, and the frequencies looked pretty random.

What I find bizarre is the pattern of the Footsie over the last 5 years. From 2013-4 there is a very strong channeling going on. It seems a little suspicious, actually. Maybe I just need to replace my tin foil hat, though.

DaveGJones

28th Feb '19

Nick, I like where your reasoning and models start in your comment, because I use the primitive form of Brownian motion in my models. My observation suggests that ultimately agent actions must agree with their computations, and their computations must agree with the underlying conditions.

Thus the “information” and “work” put into the markets should always be accessible once the signal is separated from the noise.

Ramridge

1st Mar '19

Excellent post, Timarr. More rigour in our analysis and approach to investing is sorely needed.

In order to avoid any bias I have adopted a system which is to come up with a hypothesis and then to back-test it to destruction against the full LSE equity market. I have been doing this for 12 months now.

The biggest hurdle i have come across with this approach are false positives. I'll give you a simplified example. Say I have come up with a system that traps a share that has recently rocketed. Back testing this system against the full market will give me not only this share but a lot more that turn out to be real dogs. The false positives.

My target now is to achieve at least a 60/40 ratio of "true" positives to false positives. I then apply the rule of letting winners run but cutting losses at 10% maximum, creating a bias in favour of the winners. The best i have achieved so far is 55/45.

Am I chasing fool's gold? Maybe. But my semi-scientific approach keeps my feet firmly on the ground. If back testing a potential system shows no real gains, then I ditch it and look for another hypothesis.

AnonymousUser605348

Sounds like a sound approach but one of the issues with backtesting that it always skews in favor of the past. This is a bias of all studies. For example, between 1950 and 2000, being long the S&P 500 above the 2 day MA and in cash below it, you'd have beat the buy n' hold in the vast majority of years (according to bullmarkets.co). But after 2000 it just didn't work anymore. Markets changed. The new choppy markets unwound the effectiveness of short term MA strategies. But a backtester might still look at his backtest and conclude that in the long run the system should work. It didn't.

Further, people in general tend to be arrogantly conclusive in their thinking. A liberal friend of mine once told me that he'd read a study which "showed" that immigration hadn't resulted in lower wages. Now I have no idea whether immigration effected wages or not, but the idea that such a study is conclusive is ridiculous. We can't ever know the counter-factual. The idea that an academic can tell us what would have happened had we gone down a different road is rank speculation in the guise of serious research.

My point is... you might have a lot of historical true positives that turn out to be false positives in the future. Personally, I tried technicals for years, and found backtesting provided a lot of false hope. Doesn't mean you won't do well with it of course. But be extremely suspicious of good backtest results. They are frequently poor indicators of future performance.

Lessons in Investing from Science

Unlock the rest of this article with a 14 day trial

About the Author

Timarr

6 comments

Unlock the metrics for all these stocks and more...

Unlock the metrics for all these stocks and more...