tick chart data mismatch

[hypertrader]

I was trying to set up a trading system based on ES tick data in an advanced chart.
However, I already ran into problems when I wanted to plot a simple moving average either directly as a basic study or called from an efs.
After a couple of seconds there is a drift between the tick data and the plotted MA.
The easiest way to check this is a setting the MA length to 1. This should give an MA plot connecting all marked ticks (if the advanced chart is set to candlestick) or two exactly overlapping plots (if the advanced chart is set to line). Instead, there are spikes in the MA line where none should be and vice versa. Further investigation reveals that the close or open values shown in the cursor window (which should be and acrtually are identical for tick data) also quite often differ from the return values of open() or close() when called from within an efs. The latter values are mostly in sync with the MA of period length 1.

What is going on? Any advice or ideas? Thanks in advance!

[rmclean]

hypertrader,
I was able to duplicate your problem when I set a chart of ES to a time period of 1T and applied a moving average of length 1. There is indeed a drift after a couple of seconds. However, this does not occur if I make the time period 2T or greater.

I don't know what is happening. Wiser heads than mine will have to resolve this. However, can you program around the 1T time period?

Bob

[rmclean]

hypertrader,
Correction, it is a chart of IBM and there is a drift with a time period of 2T. However, the drift does not occur instantly. It is only seen after about 30 or 31 bars print correctly. Then it is seen on the old bars, but the new bars are still printing correctly.

Bob

[hypertrader]

Hi Bob,

thanks for your input. I too did observe that the most recent tick value is correct, meaning that the MA of length 1, the high(), low(), open(), close() are identical to the price shown in the chart and the in cursor window.
However even for the next most recent bar/tick there often is a difference. The probability of a mismatch seems to increase the further you go back in time. Already after 20ticks the probability of matching data seems to be pretty slim.
This is quite disturbing, making backtests based on tick charts for parameter optimization quite worthless.

[hypertrader]

I found a thread dealing with "indicator drift" in tick charts where Jason confirms the existence of a problem:

http://forum.esignalcentral.com/showthread.php?s=&threadid=7356&highlight=indicator+drift

I wonder if this is related to what I was referring to and if the problem is being addressed?

[hypertrader]

When loaded in a tick chart (I am using ES M4) the following simple function draws the close() values as a red line. It should connect all tick marks. As you can see below the result looks quite different.

Performance should not be a factor since max CPU load was 25%.

Any ideas / suggestions highly appreciated!


function preMain()
{
setPriceStudy(true);
setCursorLabelName( "close", 0 );
setPlotType(PLOTTYPE_LINE, 0);
setDefaultBarThickness(3, 0);
setDefaultBarFgColor(Color.red, 0);
}

function main()
{
return close();
}

[hypertrader]

Another example.

This time in tick replay mode:

[hypertrader]

Just installed 7.6 (Build 636A) - no help! - ???

[Guest]

found a thread dealing with "indicator drift" in tick charts where Jason confirms the existence of a problem:

I believe your assessment is correct. I did a lot of work early on with Renko charts and they had a drift issue similar to this. I am not sure if it has been fixed, but the underlying cause for the drift issue appeared to be multiple simultanious bars updating on the chart while the efs, by design, recognized this as one bar. The output from any study would gradually drift to the left since the chart had more bars than what the efs believed.

Basically an accounting thing with respect to barcount IMO. The efs started off matched up with the advanced chart, but since the accounting of bars was slightly different between the two engines, after an hour or two of RT data, the efs believed the zero bar was (for example) bar (-10) on the advanced chart. Thus the drift was evident.

I suspect what you are seeing is something similar here, where the barcount of the efs is out of sync with the chart. however in this case, the efs sees 20 bars (for example) on the chart, wheras the graphic engine sees 35. In this case however, there are chunks in the middle that the graphic engine sees that are not being seen by the efs

This is conjecture on my part and I may be all wet, but those are my thoughts. I know it is probably over 100 times harder to fix than we may think. If it were easy, I am sure it would have already been done. Not being easy, for an effective fix, some major code work is probably required and a sound understanding of the associated impact of the fix is essential (fix 1 thing, break 10 others).

They have a lot very smart people at eSignal, I am sure a fix is in the mill.

[hypertrader]

Hi Steve,

thanks for your reply. I think you are right with your assumption.

I also do believe that there are some smart people at eSignal.

However, this alone just does not help if you want to get along with designing robust trading systems and instead keep running into various problems.

But nevertheless lets still hope for the best...

[hypertrader]

It´s been quite some time since I have posted the problem regarding the drift of data in a tick chart.

It prevents me from setting up my trading system properly.

After other users have confirmed the problem, I would appreciate if anybody from eSignal would comment on this bug:

What causes this behaviour?
Is there any workaround?
Is the development team aware of the problem?
When will it be fixed?

For some ideas about improving quality management (and reference to further bugs):

http://forum.esignalcentral.com/showthread.php?s=&threadid=8075

BTW is this bug related to the Renko chart drift problem described in the following thread?

http://forum.esignalcentral.com/showthread.php?s=&threadid=8308

Thks in advance

[DuaneG]

Hi Hypertrader,

The Development team is very aware of the tick drift issue. They are working diligently to get this one nailed down. As for a work around, one option is refreshing the chart. This is not an ideal solution to the overall issue, but until resolved, it is something to keep in mind.

To the best of my knowledge, this is somewhat difficult issue to fix, but every effort is being made to get it resolved. As for the reference to the previous threads, this is the same issue being discussed here. As we get more information regarding the status of this, we will follow up with the pertinent threads. Thanks for your patience.

[hypertrader]

Hi DuaneG,

more than 6 month have passed after your last post.

I am curious about what happened since then concerning the tick chart indicator drift problem.

May we expect a solution to this from the announced bug fixes of version 7.8?

Thks in advance for the prompt reply

[DuaneG]

Hello Axel,

Please read through the following thread or more specifically, the most recent post from JasonK. This bug took a while to isolate and fix, but everything thus far has been very positive in regards eliminating the drift.

[hypertrader]

Hi DuaneG,

thnks for the prompt reply. Sounds quite promising. Looking forward to 7.8.