Inconsistent data

[saxon987]

Recently I noticed that a trading system I use with eSignal data was telling me that I should be long; then when I would close the app and open it again (establishing a new eSignal link) it would tell me that I should be short!!

I reported this oddity to the app support, and I got this reply:

Each time you refresh the data in ESignal it causes a new request to be issued to ESignal's server farm. We've documented that these requests do not always returns exactly the same data. Usually the historical bars are very slightly misaligned, sometimes causing open/close times to be slightly different between one request to the next. Needless to say, this can cause system results to change. I've spent many many hours on this issue supporting a large trading firm in Chicago who was experiencing the same thing, so I know it can be a problem. Unfortunately there hasn't been a solution forthcoming from ESignal on this one.

BTW, you can validate this yourself. Open a chart with ESignal live data and right click on the chart and say Copy Price data to Clipboard. Paste into Excel. If you later run the system and see different results, copy the data again. If you compare the 2 data in Excel you will see the slight differences I speak of.

The cause of the problem is that different historical requests can be handled from different ESignal servers in their server farm, and we cannot force the app to use a single historical server.

What is up, eSignal?? This is a significant issue that needs to be fixed!!

Thanks,
saxon

[saxon987]

eSignal,

Is this issue being addressed, or just ignored (like my first post)?

Man-o-man...

I would have thought data consistency would be an issue you guys would take an interest in.

Maybe not.

saxon

[ScottJ]

Hi,

Looks like we just missed your earlier post. Sorry about that. This FAQ explains the issue a bit clearer. At the bottom of the FAQ, we note a network project that will implement a new time synchronization process across all our servers and improve the consistency of results after a refresh or on different PC's.

That project has proven a bit more difficult to implement than originally thought so the date has been pushed out. We now expect the new time synchronization mechanism to be in place by mid to late Jan (2004).

Thanks.

[saxon987]

Scott,

Good news. The sychronization will make things a lot less confusing.

Thanks,
saxon

[saxon987]

Zapped again...

I hope this issue of historical data consistency is a MUCHO high priority with eSignal (being in the business of providing clean and accurate data).

A discrepancy between one data download, and a second download of the SAME data cost me $2000 today!

Hoping (and expecting) that this will be resolved very soon,

saxon987

[morpheous]

Top priority

I agree, the issue of consistent and reliable data MUST be Esignals top priority. I recently lost quite a bit of change when the third party trading system I was using with esignal's data feed gave me a sell signal. After shutting down and reopening in the PM the sell signal was not there, but my trade was.. so I exited immediately with a loss. No amount of market analysis and planning can prepare you for inconsistent data.

Morph

[RobMc]

"Update: We are implementing an interim solution that will allow all of our tick servers to synchronize to a single network timestamp. This should greatly enhance the consistency of our intraday history data across our server farms. This solution should be completed by mid to Mid-Feb, 2004."



The above is from the update link provided by ScottJ below. Is there any further update on this major problem now we are in mid-Feb? Looking at a chart this morning I got a signal and checked in another timeframe but when I went back to the original chart the signal had gone. This is a major pain and also backtesting becomes a joke if the results change every time you test the same historical data.

First it was mid Jan, end Jan, mid Feb ......... please tell me it is sorted. It may be a difficult or complex problem but it is a very important issue.

Thanks
Rob

[saxon987]

RobMC

Rob,

I received this reply from eSignal on another board.


02-04-04 05:58 PM

Hi Saxon,

I received an update today that we are on schedule for the tick server synchronization to be completed by end of next week.

Regards,

Andy S.
eSignal Support

That would make the completion date...TODAY! (2/13)

I hope...I hope...

Yeah, this issue has proven to be a real pain. I just hope (and pray) that eSignal is really serious about correcting it. I'll believe it when I see it, and be a HUGE eSignal fan if they do...

We'll see.

saxon987

[ScottJ]

Update on Server Synchronization:

The good news is that we've begun synchronizing servers and the early results look very promising. We have approximately 10 servers in "lock-step" or synchronized and will continue rolling the new programming out to all the other 180+ servers over the weekend and into next week.

We elected to roll this out over 6-10 days rather than all at once so we could have the highest degree of assurance that the new code was functioning properly. As a result, we'll need to wait until late next week to declare this problem resolved.

Thank you.

[RobMc]

Scott,

Thanks for the update, good news indeed.

180+ servers??? You'll need a lot of space under your desk for that lot

Regards
Rob

[ScottJ]

Thanks for being patient!

Thanks for the update, good news indeed.
180+ servers??? You'll need a lot of space under your desk for that lot

2 Ticker plants, 4 co-location server farms and seperate servers to handle RT quotes, tick history, daily history, Market Depth, and News. It's a complex and powerful infrastructure and that complexity is one reason it sometimes takes us a while to resolve problems that, on the surface, may seem somewhat easy to fix. You're right, some of us have big desks

Thanks.