ATR for same bar changing day to day

[richthomas]

I have been running a back testing efs for symbol EEV on 10min charts for the month of October 2008. I noticed that I was getting different results depending on whether I ran it while the market was open or closed for exactly the same function parameters. I then discovered that it was varying day by day irrespective of whether the market was open or closed.

I am using variable lot sizes based on a risk value, trade commission ($13.90) the previous bars ATR and an initial stop loss multiple as follows:

nLot = Math.round((Risk - 13.90)/(islMult * nATR1));

ATR is obtained from:

xATR = atr(14);
nATR1 = xATR.getValue(-1);

Risk and islMult are passed as function variables and do not change for each run.

On comparisson of the trades of each of the BTs run I noticed that for a few of the trades there was a slight difference between the nLot sizes e.g. 186 v 190. Since the only "variable" in the nLot calculation is the atr I started checking this through.

I also have an initial stop loss and trailing stop loss that are also based on the atr value. And this contributed to some variations depending on when the BT was run.

I displayed the builtin ATR study on my chart and checked the calculations for nLot and my stop losses against this and for each run through the calculations were correct.

However when I checked the values of nATR1 on the built in study for the trades where I had a discrepancy I noticed that the values were different depending on when I looked at the chart!! (beginning to sound like quantum mechanics)!!

To be specific, using the 10min chart for EEV on 29th October at 18:20 GMT, 14:20 Eastern and built in study ATR(14), looking at the ATR study on 12 Nov when the market was open the atr value was 2.944885. However when I looked at it on 13 Nov before the market opened its value was 3.258617!!!!!

How can the atr for the same bar of 2 weeks ago change from day to day?

If there is a problem with the atr study can you recommend an alternative method of obtaining it that doesn't have these inconsistancies?

I have attached my efs for completeness, however the problem appears to be in the raw data changing.

[ACM]

Re: ATR for same bar changing day to day

richthomas
The issue you are seeing could be due to the method used to calculate the average in the ATR study which requires a certain number of bars before it is fully stabilized (you can review the calculation in the ATR.efs which is included in the Library folder of Formulas)
If each day you change the starting point of the data [and consequently the starting price] you will have a certain number of overlapping bars from the initial plot where the ATR will be different from one day to the next.
Alex

Originally posted by richthomas
I have been running a back testing efs for symbol EEV on 10min charts for the month of October 2008. I noticed that I was getting different results depending on whether I ran it while the market was open or closed for exactly the same function parameters. I then discovered that it was varying day by day irrespective of whether the market was open or closed.

I am using variable lot sizes based on a risk value, trade commission ($13.90) the previous bars ATR and an initial stop loss multiple as follows:

nLot = Math.round((Risk - 13.90)/(islMult * nATR1));

ATR is obtained from:

xATR = atr(14);
nATR1 = xATR.getValue(-1);

Risk and islMult are passed as function variables and do not change for each run.

On comparisson of the trades of each of the BTs run I noticed that for a few of the trades there was a slight difference between the nLot sizes e.g. 186 v 190. Since the only "variable" in the nLot calculation is the atr I started checking this through.

I also have an initial stop loss and trailing stop loss that are also based on the atr value. And this contributed to some variations depending on when the BT was run.

I displayed the builtin ATR study on my chart and checked the calculations for nLot and my stop losses against this and for each run through the calculations were correct.

However when I checked the values of nATR1 on the built in study for the trades where I had a discrepancy I noticed that the values were different depending on when I looked at the chart!! (beginning to sound like quantum mechanics)!!

To be specific, using the 10min chart for EEV on 29th October at 18:20 GMT, 14:20 Eastern and built in study ATR(14), looking at the ATR study on 12 Nov when the market was open the atr value was 2.944885. However when I looked at it on 13 Nov before the market opened its value was 3.258617!!!!!

How can the atr for the same bar of 2 weeks ago change from day to day?

If there is a problem with the atr study can you recommend an alternative method of obtaining it that doesn't have these inconsistancies?

I have attached my efs for completeness, however the problem appears to be in the raw data changing.

[richthomas]

Alex.
I am ensuring that I have data loaded on the chart a couple of days prior to 1st Oct. Then I always run the back test just to process bars from the 1st October through to 31st October inclusive. As there are 39 10min bars to a day, shouldn't atr(14) have stabilised by the 1st October?

It surely should have stabilised by 14:20 EST on 29th October which is the bar that I was querying?

Over the month of October I had 26 trades. Some of them are exactly the same day to day but a few are not.

[richthomas]

Alex,
I have run the same back test against the same data range (1 Oct to 31 Oct inclusive) for the same symbol (EEV) and interval (10mins) on the last three days 12, 13 and 14th Nov. On the 12th I ran it with the market open and also closed. So I have 4 printouts of the trades executed. I have used the first printout (12 Nov market closed) as the base line and compared the other three results to it.

3 of the runs have 26 trades - one of them has 27!
Each of the 3 non base line runs has 6 to 8 different contract sizes compared to the baseline. In some cases the differences are quite small i.e. 199 v 200 in other cases it is larger 438 v 445.

As I mentioned below in calculating the contract size the only element that isn't passed as a function parameter is the ATR which leads me to believe that it is changing between BT runs for the same bar - and this was validated by looking at the ATR study on the chart for the same bar on different days.

I don't know whether this has any bearing but my preferences for Date/Time settings is Local Timezone (I am based in the UK). My time template is called GMT and is set to Intraday Default, start time 14:30 and end time 21:00.

It is important that I can trust the results and that they are consistant as I have optimisation code that runs multiple iterations of DI length, MA length, initial stop multiple, trailing stop multiples and Profit targets and selects the best combination to give the best win rate and ROI combination. Since several of these calculations use ATR it is important that this is correct.

[ACM]

richthomas
I have uploaded here a zip file (EEV atr issue.zip) that contains the HTML outputs of the back tests I have been running every day at market close from the 13th to the 21st of November [there are two tests for the 21st one that was run while the market was open and the other at the close]
The Time Template used in the back tests was initially set to 60 days and incremented by 1 day each subsequent day so as to maintain the same starting point for the calculations of the ATR (note that I also ran the tests using always 60 days and the results were the same). Start/End Times used were 9:30-16:00 ET.
You can identify the days in which the back tests were run by looking at the date of the CloseOut trade (ie the last trade). This is because my Time Template ends at 16:00 and not at the time you set in the script hence it carries the last trade to the last bar on the chart
At my end the results of the back tests appear to be all the same with the exception obviously of the last trade of each test which changes daily. However even in that case the number of shares is constant in all the tests
As an aside I notice that you posted a similar query in the EFS Studies forum and were given a reply similar to the one I provided in a previous message.
Alex

[richthomas]

Alex,
Things have progressed in my evaluation of the problem since my last post and I have found a more fundamental problem. I wrote a simple EFS that wrote the atr(14) value to a file for each of the bars between 1/9/8 and 31/10/8. I then copied these values to an excel spreadsheet. Everytime I recorded the atr values I could then compare the values within the spreadsheet to see where the differences were occuring.

I also recorded the OHLC for each of the bars and did a comparison and found that there were various changes in these values! I contacted tech support in the UK and the response I have been given is that there are 60 tick servers in two different physical locations in the US. Everytime a user accesses esignal they will access one of these servers. It could be a different server each time. Apparently there are data discrepancies between each of these servers and it is too much effort and cost to try to get these in sync!!!!!!!! There are 1716 bars that I am comparing and 4 prices for each bar making 6864 prices in total. for each run there are between 10 and 220 differences, so overall a low %. Some of the differences are quite small 2 - 10 cents, others are quite large 4.32 dollars and these can throw the results each time I run my backtest!

Are you aware of this problem? Do you know why it is occurring? I thought the source for data for EEV for example would always come from the same place so the data should be consistent across all servers.

If this really is a well known problem then why isn't everyone experiencing the same problems that I am?