Evaluating claims of the hidden order of markets
Many market pundits claim that there is a hidden order to the financial markets, revealed by astrology or the interpretation of various arcane numbers, "waves", etc. Many people believe in what these "gurus" are saying. Is there any truth to it? I offer up a few ideas for thinking about these kinds of things:Testimonials are not evidence.
Most testimonials are written by people who have had one or two lucky trades after newly subscribing to a service and are all pumped up about their "success". You just don't see any testimonials by people who have 100 trades under their belt.Trading magazines are not peer reviewed, scientific journals. Neither are books.
Articles in trading magazines are mostly written by people looking for clients, customers and seminar attendees. The remainder are written by people looking for recognition. Same goes for books. Just because something is printed in a book doesn't mean it is true, or that it has been well researched.It may be useful to examine the point that even when something is published in a scientific journal it doesn't mean it has been proven true. Unfortunately, even in scientific fields, due to human nature, we must assess the author's motivations and the source of funding for the research. Who was paying for the research and what did they want to find? Was the author up for tenure? Desperate to please the people behind the grant money? In search of recognition? Planning on selling a book?
After assessing these factors, we still have a lot of important questions. Was the study done properly? Was the sample size large enough? Was the sample truly random? Was the study double blind? Were any improper assumptions made? After passing these tests in study after study, we can then assume there is some validity (until proven otherwise).
A good peer reviewed scientific journal may look into most of these questions, but even then, many of the "human" motivations do not become clear until long after the fact.
One or two events do not constitute proof. Statistical "proof" requires a lot of data.
If you say that the 84 year Uranus cycle perfectly called the 1903 and 1987 crash lows, we must assume it is by pure chance, unless you have stock market data for the past several thousand years that reliably indicates crashes each and every 84 years.
A rule of thumb in statistics is that you need a minimum of 30 random, representative samples to begin to get reliable results. But this is just a heuristic. To get an idea of where this comes from, you could use error = 1/sqrt(n), where n is the sample size (and assumes the samples are random and representative of the entire population), as a rough measure of expected sample error. The remainder, or 1-error (plotted on the graph), would give us a measure of confidence. At 30 samples (where the vertical line is on the graph), our confidence would be around 82%.
Is that enought to trade with? Well, you tell me - is it okay with you if the methodology that you developed and are about to risk your money with has about an 18% likelihood of being due to luck?
This thirty sample thing is just a guideline, typically for areas of science where there actually is a very real deterministic underlying process that is unobscured by noisy data. It is the point on the curve where we may have an acceptable level of confidence, and further increases in sample size (i.e. in academic terms, requiring more grant money) yields a diminishing return in terms of error reduction. In regard to short term commodity trading systems, you need to have a good sized sample of each and every type of market condition that will be encountered. Personally, I like to see thousands of events.
What would constitute proof?
Ideally, a "market wizard" with knowledge of the hidden order in the markets should be able to simply hand that knowledge over to an academic to evaluate. Lacking that, they should be able to hand that knowledge to an ordinary trader, who should immediately become a new supertrader. But I'm not really sure that we can find a proven "market wizard" in the first place. And then there's the problem that these folks don't want to reveal all of their "secrets", which I suppose is fair enough. Or they claim there is more to it. So, we're back to square one.And that is to say, very simply, a trader with knowledge of the order of the markets should be able to substantially and consistently outperform traditional strategies, on an absolute and risk adjusted basis.
In other words, they should prove that they are in fact, market wizards.Ideally they should use only their "secret" methodology, with no mix of trend following techniques, etc. It should be as pure a test as possible of the hidden order that they claim exists.
They should use a real money trading account, using a substantial amount of money (i.e. enough to "hurt", which of course depends on the trader but I would say 100K minimum of personal money). Participants in simulated accounts or small money accounts tend to take on too much risk in the hopes they may hit a ball out of the park. The account needs to be traded over a sufficient period of time. Depending on the time frame of the trader I would say at least 100 trades. It would be nice to get in one or two economic cycles, maybe 5-10 years. This would be a start in the right direction.
It would also need to be directly comparable to other traders operating under the same conditions. Money under management for others, such as a CTA, with all the reporting that is required, would probably be the ideal situation. Three pieces of information are needed: return, risk (such as maximum drawdown), and a measure of the return relative to risk (such as the sharp ratio).
Anyone with a real clue as to the order of the markets should be able to blow away the best mathematically oriented trend followers or value investors, time after time. I mean, if this stuff is true, shouldn't there be a sizable group of these crazies with unbelievably high returns year after year after year? I'd say a ten year track record, doubling the risk adjusted return of the best mathematically oriented trend follower in each year would be substantial proof. Frankly, someone with real knowledge of some kind of hidden order should literally be able to destroy the markets, to the point that no one would trade with them.
If they are unable to do something like this, if their overall performance is not significantly better than those using simple trend following or other conventional methods, then it follows that the hidden order they speak of does not exist, or the effect of the hidden order must not be too significant, or they are just not that good at what they are doing.