Tweet
Back Testing Trade Analysis Report grossly inaccurate: Percent Profitable reported 70% when in fact it is closer to 25%.
I noticed that when back testing any trading strategy using multiple time intervals, the profits reported are grossly exaggerated, as the equity curve looks like the "Holy Grail" of trading and when I've actually turned on the strategy for automated trading rather then being a profitable strategy 70% of then time it was profitable 25 percent of the time.
My "real" strategy used for automated trading with IB, is based on a variety of indicators, some included with eSignal some part of AGET but to illustrate the back test report inaccuracies, let's take the simplistic case of a strategy (makes little to no logical sense but illustrates the point) that buys when a close is above an EMA and goes short when the close is below the EMA.
The strategy in the EFS is using a 15 minute interval, the chart interval in the loaded chart is a 5 min, NQ #F is the symbol the EMA is accessed in the EFS via the very simple function call: ema(5,inv(15))
The first Image on the left is the Equity Curve when the chart Interval is set to 5 min, keep in mind in this example the chart 5 minute interval is smaller then the 15 minute interval referenced in the strategy.
Once the chart interval is changed to the same 15 minute interval referenced in the strategy EFS the equity curve reverses direction and is a significant looser as can be seen in the chart on the right above. It's quite a valid option for to use a shorter time interval than the intervals used internally within the strategy, as many traders do and one would expect those responsible for validating the results of the back tester would done so, but apparently not.
One can easily overcome this inaccurate back testing report by remembering that when the chart interval is less than the interval referenced in the strategy unless one insures the chart interval is the same as the intervals accessed internally with the EFS (via the ema(5,inv(15)).
Hopefully users are also aware that when running any back test that invokes the "Strategy.methods", such as Strategy.buy or Strategy.dosell, if Strategy.MARKET is used with Strategy.THISBAR the profits reported in the back test report are also grossly inaccurate indicating another Holy Grail strategy has been found.
Similar to profits reported using Strategy.MARKET and Strategy.THISBAR, there is in effect "looking back at price before entering the trade" inaccuracy in terms of the back tester using the Open rather then Close price. The final result is a even greater percent profitable reported in the back test reports using multiple time intervals.
In the example above using using multiple time intervals, when a strategy triggered using a 15 minute multiple or external to the loaded charts time interval. The back test will report the trade as being filled on the 5 minute bar, when in fact in real time it is not filled until completion of the 15 minute bar, grossly inflating the profits reports.
As in the first case when specifying Strategy.MARKET and Strategy.THISBAR, the fill price is the OPEN of the current bar, or when using external or multiple time intervals the price at the time of the lowest bar interval (5 minute chart loaded using on a 15 minute interval in the EFS). Even when the lowest chart interval is set to the same as the lowest in the Multiple Time Intervals referenced in the EFS, the fill price is reported on the lowest interval.
In the past Avery, Alexis, and many others at the time went to great lengths in helping us in the forum to overcome the Strategy.MARKET, Strategy.THISBAR issues. They would post code examples, and were overly generous in their rime and kindness in helping us out. I for one miss those days of collaboration between the eSignal staff and user community. I believe it greatly enhanced the value of the product and provided a unique competitive advantage for eSignal.
My suggestion to those struggling with developing profitable trading strategies is to validate the results in the eSignal Paper Trading DEMO Accounts.
Good Luck.
Glen
[email protected]
I noticed that when back testing any trading strategy using multiple time intervals, the profits reported are grossly exaggerated, as the equity curve looks like the "Holy Grail" of trading and when I've actually turned on the strategy for automated trading rather then being a profitable strategy 70% of then time it was profitable 25 percent of the time.
My "real" strategy used for automated trading with IB, is based on a variety of indicators, some included with eSignal some part of AGET but to illustrate the back test report inaccuracies, let's take the simplistic case of a strategy (makes little to no logical sense but illustrates the point) that buys when a close is above an EMA and goes short when the close is below the EMA.
The strategy in the EFS is using a 15 minute interval, the chart interval in the loaded chart is a 5 min, NQ #F is the symbol the EMA is accessed in the EFS via the very simple function call: ema(5,inv(15))
The first Image on the left is the Equity Curve when the chart Interval is set to 5 min, keep in mind in this example the chart 5 minute interval is smaller then the 15 minute interval referenced in the strategy.
Once the chart interval is changed to the same 15 minute interval referenced in the strategy EFS the equity curve reverses direction and is a significant looser as can be seen in the chart on the right above. It's quite a valid option for to use a shorter time interval than the intervals used internally within the strategy, as many traders do and one would expect those responsible for validating the results of the back tester would done so, but apparently not.
One can easily overcome this inaccurate back testing report by remembering that when the chart interval is less than the interval referenced in the strategy unless one insures the chart interval is the same as the intervals accessed internally with the EFS (via the ema(5,inv(15)).
Hopefully users are also aware that when running any back test that invokes the "Strategy.methods", such as Strategy.buy or Strategy.dosell, if Strategy.MARKET is used with Strategy.THISBAR the profits reported in the back test report are also grossly inaccurate indicating another Holy Grail strategy has been found.
Similar to profits reported using Strategy.MARKET and Strategy.THISBAR, there is in effect "looking back at price before entering the trade" inaccuracy in terms of the back tester using the Open rather then Close price. The final result is a even greater percent profitable reported in the back test reports using multiple time intervals.
In the example above using using multiple time intervals, when a strategy triggered using a 15 minute multiple or external to the loaded charts time interval. The back test will report the trade as being filled on the 5 minute bar, when in fact in real time it is not filled until completion of the 15 minute bar, grossly inflating the profits reports.
As in the first case when specifying Strategy.MARKET and Strategy.THISBAR, the fill price is the OPEN of the current bar, or when using external or multiple time intervals the price at the time of the lowest bar interval (5 minute chart loaded using on a 15 minute interval in the EFS). Even when the lowest chart interval is set to the same as the lowest in the Multiple Time Intervals referenced in the EFS, the fill price is reported on the lowest interval.
In the past Avery, Alexis, and many others at the time went to great lengths in helping us in the forum to overcome the Strategy.MARKET, Strategy.THISBAR issues. They would post code examples, and were overly generous in their rime and kindness in helping us out. I for one miss those days of collaboration between the eSignal staff and user community. I believe it greatly enhanced the value of the product and provided a unique competitive advantage for eSignal.
My suggestion to those struggling with developing profitable trading strategies is to validate the results in the eSignal Paper Trading DEMO Accounts.
Good Luck.
Glen
[email protected]
Comment