Wide price discrepancy between real time and back test trade prices

[demarcog]

I have a strategy that when run using the back test program reports Trade prices at one price and when the same strategy is run real time the actual execution prices differ considerably. The difference goes far beyond any slippage and reflective of a software issue.

The strategy paints the background green when long and maroon when short.

As can be seen in the image below at the 11:06 am with the background painted green and a closing price of 7225.75 the back test report reports a long trade. However when run real time as can be seen the trade does not occur until two bars later at a price of 7239.50, that is a price difference of almost $300!


The issue I am describing can be clearly seen when comparing the chart with the green painted background at 11:06am, the back test fill price at 11:06am and the actual execution several bars later at 7239.50.

Not sure why this is occurring as usually the back test reporting is an extremely useful and accurate tool however I strongly suggest it not be relied upon without verifying the results without a paper or actual broker trade history that validates the back test results.

Hope this helps.

glen

[email protected]

[ewdeg98]

When trading live are you using Market or Limit orders?

The Back-testing strategy enters a trade exactly at the price and time specified. If you're using a Market order then it is subject to the current bid / ask which can cause a delay.

[demarcog]

Hi ewdwg98,

I do use market orders and the NQ E-mini contract bid/ask spread is usually one or two ticks $5-$10 wide, the discrepancy I'm seeing as mentioned above is $300, 60 ticks or 15 Full NQ points. I always account for slippage in my back test reporting, this discrepancy goes beyond what you suggest but I appreciate the heads up. Your question raises the age old question of whether to whether to use market orders and pay the bid/ask spread or use limit orders and avoid paying the spread. Using limit orders runs the risk of missing trade entry and frequently limit orders will not be filled for trades that show the most profit. I recall when the bid/ask spread was .50 and .125 in OTC and NYSE trades now it's pennies or .05 so for liquid markets I'm happy to pay the spread to secure my position. My systems for profitable trades holds positions for minutes to hours, so a tick or two going in and out is insignificant in the bigger sceme.

I have documented several major inaccuracies in the Back Test reports that I imagine many are using to base trading decisions upon, and also opened issues the eSignal customer technical support. What's beyond troubling is that it has been several weeks and I have not heard back from eSignal technical support either by email or in response to the forum post, very unusual as they are usually extremely responsive.

It is disconcerting that eSignal owned by one of the largest, most innovative and successful exchange conglomerate's in the world is apparently not allocating sufficient support resources as they have always done quite substantially in the past.

I have been an active eSignal user for decades, consider it the premier software for trading, testing and building automated trading systems and have recommended it regularly to many firms.

The apparent recent lapse of support in this one specific EFS back test area concerns me greatly and I am hopeful and confident eSignal will provide the resources available to prioritize the issues I documented. There are incredibly talented people at eSignal that I have had the pleasure to work with over the years: Alexis, Avery, Luis, William, Alec, many amazing developers and world class technical and sales support staff.

For whatever reason the firm seems to not be allocating these invaluable resources where they are very much needed by myself and others and hopefully the situation will be rectified quickly so I can continue recommending the product, and resolve this simple back test calculation issue. When one time interval is used to back test, the application is dead on accurate. When multiple time intervals withing the strategy, or when the chart interval is smaller then the EFS time interval the results could not be more inaccurate, always showing fantastic profits are not even close to being accurate in real time trading.

Thanks for your reply, hopefully others will begin to use the forum as it benefits the user community and eSignal.

glen

[email protected]

[ewdeg98]

Do you have any logs?

Right before you enter both a Back Test entry and Real Time entry, log the current time (bar), position, price, contracts etc ( everything relevant about the trade)
Compare your logs against the Back Test report and Real Time entries.
If the logs match the Back Test exactly then the problem is not with the Back Test, there must be a logic flow bug within your code relating to submitting the live orders since you are running 1 min bars, so a market order shouldn't take minutes to fill.

When you look at your backtest is it correct? Does it follow your trading logic?

[demarcog]

ewdeg98,

Thanks for the response I log and checked everything. I also paint the bar green when the EFS is long and maroon when short. As you can see on the chart the buy/sell goes out real time several bars after the bar is painted green or maroon. The color change aligns perfectly with the back test report.

You are correct 99 percent of the time errors are "user" errors however the painting of the green bar on the back test bar aligns perfectly with the back tester and the BUY @7729.50 three bars later aligns perfectly with the Account Manager generic broker function calls, that was the point I hoped my post would clarify/document as being a clear example of the incorrect eSignal Back Test report.

[ewdeg98]

Are you issuing your Back Test order (Strategy.doLong) and Live order (Trade.buyMarket) at the same time? (ie both on bar #100 for example)

You're saying the colour change aligns perfectly with the Back Test report, if that occurs right when you issue your Back Test order, then how is the Back Test report incorrect?

Lets say on bar #100 you issue a Back Test order and live order. The Back Test order is filled right away, the colour change occurs and it is shown on the report. Then a few bars later the live order is filled. To me that shows the Back Test is correct, for some reason the live order is being delayed.

[demarcog]

ewdeg98,

From your mouth to God's ears as the saying goes, I wish the back test was correct and there was a delay in the real time order.

My strategy is using multiple time frames so it appears the back test is somehow "looking ahead". If you have any strategy that uses an indicator on a higher time frame. For example a simple Price above an 15 minute ema(20,inv(15) . When you load this strategy into a 15 minutes chart and back test it it may show a percent profitable of 40%, when you keep the same strategy loaded and change the chart interval to 5 minutes, the winning percentage jumps to 70% when you change the interval to 1 minute it's over 90% clearly next to impossible.

What's I think is happening is the back tester is filling the trade at the 1 minute bar 15 minutes prior to the actual crossover occurring, "looking ahead" in a sense.

Because when the trade is run in real time the trade doesn't actually go out until the real time price is above the ema on the 15 minute chart.

I'm hoping eSignal can resolve the issue as it is very easy to reproduce.

Thanks for the input any other ideas are welcome.

glen

[email protected]

[ewdeg98]

"What's I think is happening is the back tester is filling the trade at the 1 minute bar 15 minutes prior to the actual crossover occurring, "looking ahead" in a sense."

Let's say the 1 min bar 15 mins prior to the actual crossover is called Bar A
When its being filled at the Bar A how are you comparing the price to the ema?

var myEMA = ema(20,inv(15);
if(high(0) > myEMA.getValue(0)){
// Enter Trade
}

If you're using Market Orders then the only way the Back Tester can enter a trade at Bar A, is if your code explicitly enters a trade at Bar A. I think you're right with the problem being forward looking, but it doesn't make sense that the Back Tester is forward looking since it does what it is told, rather something in your logic is forward looking which then submits a premature trade to the Back Tester.

[demarcog]

ewdeg98,

I really appreciate your help on this and I hope it is something I am doing.

The strategy is obviously more complicated but this version is a simplified to easily illustrate and reproduce the problem.

Here is a snippet to hopefully answer your questions. I don't think this will generate a "premature" trade and only familiar with Strategy.MARKET. and Strategy.THISBAR with indicators using a bar index of 0, which gives the open price of bar as the entry price, and therefore "premature" and inflates profits enormously.

The efs is hundreds of lines and I use functions like GoLong() but copied in the relevant lines into the snippet. If this isn't enough please let me know.you can email me if it's easier.

I'm basically doing the same things as you except storing the ema value in a variable which shouldn't make a difference as I copy in a new value on each new bar.


if (getBarState() === BARSTATE_NEWBAR){ // only execute remainder of EFS script once on "NEWBAR",
if (getBarState() === BARSTATE_NEWBAR && getCurrentBarIndex() === nRTBarIndex){ bRealtime = true; }

xint1ema = ema(20,inv(15));
nint1ema = xint1ema.getValue(0);

if (close(0, inv(15)) > nint1Sma ){

///// GoLong(); relevant code from function copied below, actual functions obviously has a lot more logic regarding real time trade processing but essentially this is it

if (nBarState == BARSTATE_NEWBAR && !bRealtime && !Strategy.isLong()) {
Strategy.doLong("Long Entry", Strategy.CLOSE, Strategy.THISBAR, nQty );
}
if (bRealtime && nBarIndex == nRTBarIndex && nBarState == BARSTATE_NEWBAR){
buyCallback();
}

Thanks again please let me know if you can think of anything. If you load this up I think you will see the problem I am having, keep in touch.

[email protected]

[ewdeg98]

What does nRTBarIndex store? When does it's value change?

[demarcog]

That is 0 and remains 0 throughout the code,

As I'm sure you know if you want to use one EFS strateg to invoke the back testing functions for a back test and then switch to executing the real time broker functions you have to determine when to execute which functions, The way I determine realtime is at BARSTATE_NEWBAR when the BARINDEX of 0.

[demarcog]

I reported this bug in the back tester 3 months ago and never heard back from eSignal. Is anyone looking into the problem as there are some of us who actually rely on the back tester being accurate assist us in making real life trading decisions and are paying for support and software that does what it advertises to do.

Thank you..