Back Testing strategies are FLAWED

[alias]

99% of all the back testing strategies at this site, at eSignal site
and eSignal Central at wrong, wrong, wrong.

You can't exit a trade based on the opening price using closing
information. Thats like being able to call your broker at the end of
the day after you see the Monday October 7th 2002 crash and saying
that you want to exit at the opening prices instead!

I have pointed this out to the experts at eSignal and they still
can't figure it out. Simply CLUELESS. I wonder if it isnt the 'phone
support experts' that handle the emails. Becuase they don't
understand the problem and have created all their backtesting
strategies on this principle. And so has all of the other sites that
use eSignal EFS's.

All you have to do is exit the position using with a .CLOSE instead
of a .MARKET command instead. Or you can use .MARKET with .NEXTBAR
isntead to exit at tomorrows opening price.

The thing that concerns me is eSignals sever lack of understanding
of this concept. They NEVER should have called it .MARKET when it is
really the opening price that gets returned. If you want the current
price you have to use .CLOSE. That is the most recent price.

So, bottom line, when you look at those Back Testing Analysis
Reports and see all those high dollar figures dont believe it. It's
flawed and you will loose all of your money trading that system.

Try it and see.

[Freeman]

Are you sure they misunderstand?

Do you think they'll sell more copies with strategies that show amazing backtesting profits, and to the undiscerning eye look valid when compared to the charts? When Joe-Trader-with-dollarsigns-in-his-eyes runs a backtest and finds his holy grail, the first thing he'll do is brag to his buddies about this amazing new charting software that will make him millions.

I think they understand all too well.

[AndyS]

Hi Alias,

Thank you for your post. The formulas we have included with the system are strictly sample formulas. They are intended to be adjusted and tested, just like the feature implies, back testing. In our rush to put together some sample formulas for the release we did code the formulas with, “Strategy.MARKET” rather than, “Strategy.CLOSE. This was an oversight in our rush to get this out. We will revise the default value to be Strategy.CLOSE in the next release. If you go to the following link, you’ll see that code exist for entering / exiting at open, close, limit, & stop, as well as, choices of current bar (Strategy.THISBAR) and next bar (Strategy.NEXTBAR). http://www.esignalcentral.com/training/advcharting/BackTesting.asp.

In regards to Mr. Freeman’s post, we do not advertise the use of these specific formulas to, “sell more copies” of our software. We advertise the ability to test "your" trading strategies. We know how sophisticated our clients are and would not try something so easily transparent.

Regards,
Andy S.
eSignal Support

[ROCKIN]

destation and alos do them reliably so that one can rely on the historical results given.

I am bothered however by what I read here. The only way that esignal will be accepted by us who build systems, is merit the reputation that the progam will display correct historical results that one can count on.

Historical results do not gurantee future performance, but relying on historical testing that is inherintly flawed will prove financial disaster to those trying to trade with it.

I notice that esignal also has a "testing site". If one could use that to test strategies in real-time, it would be a very big help so developers could respond to esignal so that they fix the problems, assumming that they are willing to actively support this new product.

However, asking your users to "wait until the next release" to fix a problem with the way stragies are coded is not enough support for those of us who are looking for a new platform to use. Esignal must post fixes for problems soon as they are dicovered and fixed, is what teh development community will want.

The ROCKIN

[DionLoy]

Rockin: The 'fix' only applies to the sample formulas included. There is no problem with the historical data or the engine that performs those operations.

Instead of using Strategy.MARKET combined with Strategy.THISBAR, simply use Strategy.MARKET with Strategy.NEXTBAR (if you modifying one of the sample formulas).

[ROCKIN]

Can you answer if the back-testing software acturately reports activites intra-bar. for example, if one was using a 5 minute bar on the es, and a profit target or trailing stop occured intra-bar, does the software keep track of ticks and prices intra 5 minute bar sot that it knows the order of the trades intra-bar so the profit target or trailing stop pice activations are acurate?

Rockin

[DionLoy]

Hi Rockin, sorry for the late reply...

The backtesting engine only uses the data provided by the interval, so if you're back testing on 1min chart, it will not use tick data to determine whether the stop or limit came first.

If you want to use tick data for backtesting, you will have to use a tick interval such as "T".

Dion Loy
eSignal, Senior Software Engineer

[EdKoF]

First trade of a test

If you test a strategy, for example a very simple Cross MA strategy, the system goes already long or short on the first bar of the chart, depending on whether it is above or below the MA (in this example).

No need to explain that this first trade is usually highly unprofitable, disturbing the results of the test.

Is there a way to leave this trade out?

Thx,


Edo.

[gspiker]

If you code it correctly, this isn't a problem. A simple example:

Price MA cross over:

If I only check if price is above or below the MA, then I will see the problem you describe

If I check for changes from the default state as defined by the initial bar (Price above MA or Price below MA) then I will not see this problem, since the first trade will then take place only after the state (Long or short) has changed.

[vladar]

My code is updating the chart on ticks

Leon,
you wrote:
The backtesting engine only uses the data provided by the interval, so if you're back testing on 1min chart, it will not use tick data to determine whether the stop or limit came first.


I have the following code in a backtest:

if(Strategy.isLong()) {
if (close() <= curr_stop)
{ // STOP hit - exit trade,
Strategy.doSell("Stopped-Out",Strategy.MARKET, Strategy.NEXTBAR,2);
delta = trade_in-close();
total_loss = total_loss + delta;
debugPrint(" Stopped out long at="+close()+" trade = "+delta+"\n");
debugPrint(" Total Loss = "+total_loss+"\n");
drawShapeRelative(0, low()-.5, Shape.UPARROW, null, Color.white,Shape.ONTOP, getValue("rawtime") + "B");
return;
}

And have it running on a 3 minute chart. The output window, from the debugprint statements, is showing me 'stopping' out on every tick once the stop level was hit, the chart has also printed out four white arrows on each of the candles displayed. Once I reset the chart by changing the time interval to 5, then back to 3 minute, the white arrows are gone, and all the data on the output window is back to 'normal'.........but during the 'stop out' period, if I just sat and watched the chart, it continued to show me 'stopping' out on every tick.


I'm running build 513.

Thanks.
V

[vladar]

clarification

It wasn't clear in the last note: I'm using candlestick charts, not tick charts, yet all the debugprint lines are being printed tick by tick.....

From your note I assumed that the calculations would be done at the end of the interval that he efs. is running on.

In other words, if a condition is met (such as a moving average crossing another moving average), drawing a shape or text onto the chart will not be done until the period closes.

But what happens is that the drawing is done during the formation of the candle, meaning I can get both a short signal and long signal 'arrow' displayed on the chart at the same time as the 'condition' moves back and forth across the signal value.

[dsnyc]

Vladar,
Try the new "compute on close" option in efs settings. This may solve your problem.

[vladar]

Thanks for the idea, doing that gets the backtesting reports to pop up like before.

Cheers.

[Phoenix]

Backtesting Code for Moving Average Crossovers

Can anyone point me to an efs file for backtesting a moving average crossover strategy (one MA crosses over the other, versus price crossing an MA).

Thanks,

Phoenix