I have always been convinced about the value of backtesting, yet a simulation may have a hard time taking into account every factor occurring in real trading. For example, a system may produce a certain number of trades, in backtest, some of them winning and some losing. Then you put the system working for real and it gets filled on only a percentage of them and "strangely" getting more of the bads than the goods. So even if slippage and other conditions are favorable the system makes money only in theory. I noticed this may depend on market type, liquidity, order type and other such things that are more difficult to take into account in testing. acrary (and everyone else), can you tell if these have been important aspects in your testing experience and, if this is the case, what is your preferred way of dealing with them? By the way, many thanks for reopening this valuable thread. -GS
Consideration of fills has been very important in my work. If my expected edge is not 2-5 x greater than the cost of participation (slippage and commission) I won't trade it. Obviously that cost varies with each market and the size of the transactions. Also, the higher the frequency of the system, the less I require of it.
these are all questions of frequency. my highest frequency so far is about two trades per day per market. five minute bars and in the testing i assume i get the close of the bar succeeding the close of the signal bar. at that rate i do not have that problem. nevertheless i am movin' towards higher frequ ... so i have to prepare ... do you guys know the sharpe ratios of your high freq stuff? alan, are we intervening too much on your thread? i would not mind to discuss that elsewhere in case anybody feels it' misplaced ...
Hello Acrary, Have read through this thread and am interested in the way you are combining systems. I have a question about the validity of using correlation relationships between trading systems when trying to combine systems in a portfolio. In standard portfolio theory where assets are held, portfolios can be optimised using the covariance matrix. The covariance matrix represents the correlations between the different assets. Since these correlations often arise from fundamental factors there is a reasonable chance the relationships may persist into the future. However, there is not necessarily any stable correlation relationship between the equity curve of a system and the price of the asset it is traded on. The higher the frequency of trading the less likely it is that the equity curve will have any correlation to the price of the asset. Hence, I cannot see any fundamental reason why there should be any stable correlation relationships between trading system equity curves. If there is no reason to believe the relationships will persist then optimising a portfolio of trading systems using correlation relationships seems like an exercise in over fitting. Do you have any thoughts on this?
I'm here. Just not up to doing much writing right now. I've been analyzing the past posts with a objective view and I'm pretty bummed at the quality of the material I posted. As soon as I'm ready to invest a great deal of time I'll let loose.
i dare commenting, though you addressed alan. first of all, even in standard usage of portfolio theory the instability of correlations is a significant concern. actually i would argue almost contrarian to your point: i do not see reason why correlation of trading systems should be substantially less stable than the corr between any two stock or futures contracts. two trend followers will correlate (just try to build to uncorrelated and you will know). a trend follower and a contrarian won't. why should that persist less than the relation between two "real" world time series? we did an analysis on sp500 stocks and all their corrs between 1980 and 2004 - in general positive, but nonetheless unstable. the problem of markowitz is in my view another one: correlation is not what you are concerned as a money manager, it is stress correlation that burns your guts. as long as every curve goes up you do not care if they correlate by 0.5 or 0.7, but once a system goes south you want another to show corr of as negative as possible. markowitz corr does not cover that at all. in my view it is better to take two systems, run your figures seperately on them, then combine the two by whatever algorithm and run the figures again. all your corr effects are in detail embedded. if you run corrs in between you add an unnecessary abstraction level, which destroys much information (eg the stress corr). consider this. you have a portfolio consisting of a south american coffee stock and deutsche bank. usually they will have quite low corr pretty much near 0.1 or 0.2, i'd say. now consider stress in US equities. SP down 150 points. now you have a (stress) corr of 1. consider a second portfolio of again the south american coffee stock and the german bund. they will usually correlate quite low as well. again stress in US equities. but now the stress corr is -1. exactly what you want as a money manager. markowitz can't handle this. long term in both cases the same - outcome of portfolio optimization the same. the corr swallowed the info on stress corr. (i am aware that the cases are quite simplified ) finally you are long two systems, not the correlation between them. and hence, corr as a concept has some pretty unintuitive properties: series1 series2 10% 10% 9% 9% 10% 10% correl 1,00 series1 series2 10% 10% 9% 1% 10% 10% correl 1,00 series1 series2 10% 10% 10% 1% 10% 10% correl 0,00 just 2 cents.
I know you're an old (semi-retired) pro, but no way a neural network hits > 70% consistently predicting direction of next day's close. You can take tick data as input or any combination of market sentiment on any timeframe... it's just not possible. Add 'classic' patterns like key day reversals, etc. and you still won't reach 70%. In fact, I have a means of a obtaining a 1:1 risk-reward by properly selecting the direction of next day's close for major US indexes, for almost as much $ as you want to put on it. The bid-ask can get wider once we're in low-to-mid six figures, givng risk-reward of about 1.25:1. To clarify, all you need to be correct on is if the next day's close to be + or -. Doesn't matter if you're correct but only 1-tick or 1000-ticks. All that matters is direction of settlement next day's settlement is accurate. This is a special derrivative I have access to and it's a gold-mine for me hitting just 62% correct. But even at 62% I can not be nearly as aggressive due to MM risk and also the possibility of market phase shift which can temporarily put you at 50% or lower. By the time this is detected and model retrained, I'm risking a larger % per trade on my MAE's so net profit goes down. The concept of "direction" changes when as a trader you don't care about stops, profit targets, etc. All you need to do is be right by 1-tick or more settlement-to-settlement. It's effectively a 1-day option with fixed payout (all-or-nothing). The price of option also adjusts wiht off market activity, and spread is wider in evening. I usually have to buy/sell within 45 minutes after closing or within 2-hours before opening (but price has usually moved some overnight so my neural network output is off a little, unless adjusted to add overnight input). If you're seeing > 70% with any consitency (i.e. avg that over say 100-day period before having to adjust), we can make $100M in 24 months, and I provide all the capital.
Thanks for responding. I agree with everything you have said. To add to your point about trend followers being correlated if the underlying asset is correlated. This is true, however I have found the equity curves resulting from a short term trend following system to be much less correlated than the assets they were traded on. Small differences in the statistics of each series can result in quite different trading patterns. For the case of different high frequency trading systems traded on different markets I take a starting point that all systems have no correlation at all. This seems to be a good approximation and seems to result in more stable and robust portfolios that trying to consider past correlations during the design. Your point about markowitz and its failure during market events is true. However, I have found it useful as a starting point to allocate capital beyond which further analysis needs to be performed to estimate the risk of the portfolio given certain events.