strange charting problems

[Beldin]

Looking forward to a fix...

If I'm generating signals based on a moving average or other historically based signal wouldn't that make those inaccurate as well if the previous data is not reliable?

Is this also related to getting different data sets on different loads of the eSignal software? I've saved the data from the load that looks "right", and will save another when it comes up wrong to compare.

Stephen

[GraemeB]

Hi

Thank you for your post.

If you are looking at data on a chart, like the 720 etc, then the MA will be out by a bit. But on the lower time framed charts the data is correct, it is just how the charts are constructed on a interval over 240 minutes, which is the problem.

I am sorry but the second part of the question I am a little unsure of, what was wrong with the loading of the data? Regarding other saved data which was correct?

Thanks,

[Beldin]

It seems that in the last couple of weeks I've been getting inconsistent data when loading eSignal. Mostly I notice it on the 240 minute GBP@GAIN A0-FX chart.

I load eSignal and there are gaps all over the place and the data only goes back to June, close the program and reload and it shows with fewer gaps and data back to May 1st.

This makes backtesting nearly impossible as I don't know which data is accurate, except that I was consistently getting the data to May 1st during the summer so I assume that's more accurate.

I've saved the data using "Data Export" with the correct looking data set. Once I reload and get a set that is incomplete I can save that as well to compare.

Stephen

[oortcloud]

Hi GraemeB,

Thanks for looking into the problem. I do want to bring up one more important point. Smaller time frames seem to also be effected, not just 240 min and higher.

For instance, if I load a 30 min GBP A0-FX chart, and scroll back to Sept 6, I see a sudden 3 bar gap much higher than the rest of the surrounding bars. Then if I put my crosshairs over them its really strange. Here are the times of 5 consecutive bars (the 3 bars gapped and the bar before and after the gap).

9:00
1:00 (gap bar)
1:30 (gap bar)
2:00 (gap bar)
9:00

They are all from Sept. 6. I understand you are working on this issue but I just wanted to point out that it seems to effect a lot of time frames not just the larger 240+ ones.

Regards,
Mark

[Beldin]

I just had a look at the saved data from the "good" data and "bad" data that's been loading. With just a quick look, I've found the following:

1) Duplicate bars with different high-low-close data. For example:

Bar 1 - 05/17/06 20:00:00 1.8836 1.8845 1.8806 1.8815
Bar 2 - 05/18/06 00:00:00 1.8816 1.8907 1.8813 1.8866
Bar 3 - 05/18/06 04:00:00 1.8867 1.8886 1.8829 1.8875
Bar 4 - 05/17/06 20:00:00 1.8836 1.8844 1.8835 1.8835

2) Incomplete data. I get 645 bars with the "good" data and 475 with the "bad". There are missing bars as well.

3) Extreme inconsistency in the high-low range. A quick glance shows bars out as much as 50 pips on the 240 minute chart. To get an accurate comparison of all 475 data bars would take too long for me to consider doing simply due to the gaps in the data.

Is anyone else seeing this?

Stephen

[Beldin]

Here's a visual of a data gap. This is on the 60 minute GBP A0-FX chart...

[SafwanHak]

Hi Beldin,

I agree, I've hit this problem with 1, 3, 5 and 15 minute charts and it's data within the same day, not even that far in the past to be ignored in technical analysis.

Thanks,
Safwan

[eSignal_Todd]

Thank you for the examples and continued feedback on this. As we've been upgrading our tick servers in preparation for the upcoming 8.1 CID (Corrections, Insertions, and Deletions) release, this issue has cropped up and we have engineers currently working on a fix. I've been using the examples from this thread as well as ones I've been able to come up with to give the engineers more information to help them take care of the main problem. We're hoping this won't drag on, but there isn't a solid resolution yet. They have been creating updated builds of the server software on an internal server for our QA and Product groups to point to and test, but the most recent update still shows gaps in the data as well as occasional out of place bars which have also been reported.

As soon as we have a stable release that has been completely tested, the process of upgrading the servers will start. I'll post follow ups on this thread as I have more information.

Another point I want to mention is that there are 2 separate issues that can get confused with each other and I wanted to separate them here. The first issue is the bar retrieval issue which is causing the gaps and out of place bars. All the correct data is on the chart servers, but the bar information that's being passed to eSignal is getting built incorrectly which causes some bars not to have the full interval's amount of data. While you can see this occasionally on something as low as a 2 minute interval, it's much more obvious on larger time frames because there's more potential to have missing data.

The second issue relates to how bars can get altered due to a large time frame that causes a bar to cross midnight (Pacific Time) and get cut off. This is also something we're working on at the server level, but a bigger issue due to the nature of 24 hour trading and users in time zones all over the world. There is also a work around that I believe has been mentioned before several times in other threads about using a time template to offset the start/stop times of an interval so that you have an even break of bars at midnight and don't have any bars that carry across that time. This isn't a problem for standard lower intervals as they usually cross midnight evenly, but becomes an issue as you use an interval greater than 1 hour or use any odd interval such as 13 or 17 minutes. If I find a post where this is explained better, I'll edit this post and include the link.

Edit:
More information about the second issue here.

End Edit:

I hope this clears things up a little bit, and I apologize for the inconvenience this has caused you. We'll get things fixed as soon as possible. One positive thing is that this won't require the users to make a software change, as the fix will be made at the server level.


Thank you,

[oortcloud]

Has anything changed server side within the last week or 2 in regards to this issue? The reason I ask is that I am suddenly having bad problems with 240 minute forex charts. See image below.

The 240 minute chart is one of the most important charts I use so this is a serious issue for me.

I am in the EST time zone and use a time template of 3:00 - 3:00.

Regards,
Mark

[oortcloud]

240-minute charts fixed!

I thought I would report this workaround in case someone else is having problems with gaps in thier intraday time frames. It could be your timezone setting within windows.

For several months I've had problems with 240-minute and higher intraday charts. The time template feature did not help either. I use forex data and the multi-hour charts are important for my analysis, so this has been a big pain. I almost switched to another data service because of it.

I travel between the USA and Brazil and I finally figured out than when I am in Brazil the charts did not look right. They had gaps all over the place. When I'm in the USA they look fine.

The solution was to change my timezone in Windows to "(GMT-03:00) Greenland" instead of "(GMT-03:00) Brasilia". (Brasilia is the capitol of Brazil.) I'm using Windows XP Pro with all of the latest Microsoft updates.

Now the charts look perfect in all time frames, and my clock displays the right time. I'm guessing there is a bug in either Windows (or possibly eSignal) that improperly deals with the Brasilia time zone.

Regards,
Mark

[RuGO]

I would think this issue has to be at the top of the priority list if eSignal is interested in retaining the FX crowd. The gaps in the data is only part of the problem, having faith in the accuracy of a candle formation and hoping the data is not overlapped or innacurate is another.

How much patience will it take to get our money's worth?