Doubled entries in T&S and charts

[jay60]

Here's a problem that never gets solved permanently.

Attached is a copy of the T&S for JGB10 U0-TPX on the left and
the same after a refresh on the right.
There are a number of entries that appear twice, and a few that
were missed or were wrong. In total these make up about 20% of
the trades made during this 7 minute period (a very dull market).

Doubled or Missed/Wrong
-------------------------
10:36:05 10
10:35:08 1
10:34:55 1
10:34:54 5
10:34:52 20
10:34:51 25
10:33:23 10


While I don't really use T&S, these doubled entries will show up in the volume in interval charts. The above number of contracts in this example is small. But there are many cases where 100's of contracts are entered twice on 1 minute charts. That is a significant amount for this contract and, of course, ruins any volume analysis.

The doubled entries occur sporadically, and in both fast and dull markets. If you open T&S now, you will not see the doubled entries, as a new or refreshed T&S or chart shows the correct figures.

This problem of doubled entries for TSE and OSE products has been going on for years now. Your people in Singapore have fixed it numerous times. But the fix only lasts a few days or a number of months at best. The last fix was done the middle of last month. Maybe it can not be fixed permanently, or maybe your people in Singapore need your guidance on how to really fix this.
I don't know. Whatever, would you please take a look at this and see what can be done.

Thank you very much.

[eSignal_AveryH]

Jay,

We've setup some test machines to try and better capture the duplications you've reported. Over the next couple of days we'll continue to look for the duplications. If we see anything we'll forward them on and see what need to be done to prevent it from happening in the future.

[jay60]

> Avery

Thank you for looking into this.

Actually as there was no reply to this post I contacted MichaelA on 7/9/10 and he sent it to QA . Please see <<#90185-1071949#>>.

As of late the problem has been occurring off an on as follows:

7/12 ok
13 had doubled data
14 "
15 not sure
16 had doubled data
20 "
21 ok
22 ok
23 ok

I don't stare at T&S all day so I can't catch all cases. But the next time I do I'll send you a screenshot so you can compare it with what you have.

Thank you again,
Jay

Originally posted by eSignal_AveryH
Jay,

We've setup some test machines to try and better capture the duplications you've reported. Over the next couple of days we'll continue to look for the duplications. If we see anything we'll forward them on and see what need to be done to prevent it from happening in the future.

[jay60]

>Avery

OK JGB10 1!-TPX is showing doubled data today. In the attached screenshot please see the 2 entries for 3 contracts at 09:35:22 and then on the right after a refresh only 1 entry. There have been a number of such entries this morning, such as 7 contracts at 09:13:08. These are a small number of contracts but it often occurs for 100's also. And these entries are all shown on the charts which can lead to mistaken analysis.

Please remenber that you will not see these doubled entries if you now open a new T&S. So I hope what you said meant that you have a T&S running.

Thank you,
Jay

[jay60]

> Avery

I just noticed the additional doubled data entry for 1 contract at 09:35:19.

Jay

[jay60]

>Avery

Here are 4 more in JGB10 1!-TPX:

10:07:55 4

10:07:56 1

10:08:11 24

10:08:13 5

This is all I have time for to report today.

Hope this helps,
Jay

[jay60]

>Avery

Here are two more cases I just happened to notice at 10:57:12 and 10:57:31.

I hope the examples I've sent today are enough for you. If you need more, please let me know.

Jay

[jay60]

>Avery

As I mentioned, in addition to doubled data, missed ticks are also common, even though not occurring as often as doubled data.
While I'm sure nobody expects perfection, and a missed tick here or there is of no real conseqience within a bar, there are times when it counts.

I happened to notice a case today where not just one but a number of ticks were missed in a row in JGB10 U0-TPX.
At 14:06 I noticed 22 contracts traded on IB's BookTrader that were not shown on the chart or in T&S. At 14:07 I refreshed T&S and found that the 4 trades between 14:06:30 and 14:06:42 had been missed. As you can see in the chart there is just 1 plot at 141.86 for the 10 contracts at 14:06:01. As with T&S you will not see this if you open a new chart now.

This coupled with the doubled data also occurs in TPX 1!-TPX, Nikkei futures, and when I checked last year, also in TSE stocks.

Hope this helps,
Jay

[eSignal_AveryH]

Jay,

Thank you for the many screenshot, the tests on this side are not displaying the same result as you.

It's been recommended that we if possible we get you layout which you can email to [email protected] and your location so that we can setup machines to best mirror you.

Our last thought is that I make adjustment with my schedule and set up a time during the Tokyo's trading hours so that we could review the number together. If you have Skype we can call you or I can setup a conference bridge. Please let me know if any of this suggestions work for you.

Thanks in advance,

[jay60]

>Avery

Thank you for your continued attention to this.

I think that sending you my page file would be a waste of your time as this problem has been replicated by your people over the years, and as a result Singapore has fixed it temporarily more times than I have fingers and toes. So it's not a problem at my end. And FYI, while praying for a miracle, I tried 11 a few weeks ago and also got double data with it.

Now, I'm going to guess why you aren't able to replicate the problem 'as this has happened so many times in the past', and
I'm sure it's always due to my poor explaination. So I'll try to be as clear as possible.
The reason is that you are opening T&S 'after' the doubled data is shown. You 'will not' see the doubled data if you open a 'new' T&S. And you 'will not' see the doubled volume on the a chart if you open a chart 'after' the fact. A newly opened T&S or chart will show the correct data. So, the 'only' way you will see the doubled data is if you open a T&S 'before' the market opens and let it run all day.
If you are able to let a T&S for JGB10 1!-TPX run at night there, please let me know and as I did yesterday I'll find examples to send you for you to compare to your T&S.

However, as I've said, this problem occurs sporadically. So it might take a couple of days to catch it.

Thank you,
Jay

Originally posted by eSignal_AveryH
Jay,

Thank you for the many screenshot, the tests on this side are not displaying the same result as you.

It's been recommended that we if possible we get you layout which you can email to [email protected] and your location so that we can setup machines to best mirror you.

Our last thought is that I make adjustment with my schedule and set up a time during the Tokyo's trading hours so that we could review the number together. If you have Skype we can call you or I can setup a conference bridge. Please let me know if any of this suggestions work for you.

Thanks in advance,

[eSignal_AveryH]

Jay,

I understand your skepticism with regard to us not seeing the duplicate entries while you and Singapore can. I also understood you when you say that we would not see the issue if we opened the T&S windows after the fact. Other than to check the quotes and compare them to your screenshot I have not used the test environment I set up on Monday.

With our times being offset I've figured out a way to record eSignal time and sales and a Tick Chart, I'll also try and go in and check to make sure everything is going as planned.

In the morning I'll once again check for duplications again. The only difference between today’s recording and the others is I've changed the computer time to the Osaka/Tokyo Time Zone so that we can see if it's a time issue.

AveryH
eSignal Support

[jay60]

>Avery
Great!! And thank you for all your efforts to get to the bottom of this.

I don't know how you plan to tell if identical entries are real trades (which often happens) or doubled data. Maybe you are going to compare T&S to T&S from Reuters or some other vendor.
Anyway, I'll follow T&S here today and when I find doubled data I'll send you the info which might make it easier for you. I don't know how you've set things up for this test, and excuse me for saying so, but please make sure that you don't refresh the T&S, as the doubled data will disappear.

As I've said, this problem happens sporadically, and with bad luck today's data will be perfect for the test. Hope not!

Thanks again,
Jay

Originally posted by eSignal_AveryH
Jay,

I understand your skepticism with regard to us not seeing the duplicate entries while you and Singapore can. I also understood you when you say that we would not see the issue if we opened the T&S windows after the fact. Other than to check the quotes and compare them to your screenshot I have not used the test environment I set up on Monday.

With our times being offset I've figured out a way to record eSignal time and sales and a Tick Chart, I'll also try and go in and check to make sure everything is going as planned.

In the morning I'll once again check for duplications again. The only difference between today’s recording and the others is I've changed the computer time to the Osaka/Tokyo Time Zone so that we can see if it's a time issue.

AveryH
eSignal Support

[jay60]

>Avery

Here you are. DD at 09:34:24 67 contracts , 09:39:27 33 contracts, 09:39:19 5 contracte, 09:39:17 20 contracts. The T&S on the left is after refreshing. You will notice that the symbols are different. This is just because if they are the same, when you refresh one the other also refreshes.

Another I caught was 09:15:20 for 6 contracts.

Don't know how many examples you need but I'll send some more later.

Jay

[eSignal_AveryH]

Jay,

I'm going to stop my recording now if you could PM your username so that I can now start a new recording connect to the same servers you're connected to.

I'll also send you a link of the recording from open up until we restarted which is about an hour and 17 minutes.

AveryH
eSignal Support

[eSignal_AveryH]

Jay,

Finally we have it. I connected to a server you were previously on and see the duplicates. Starting at 10:38:06 (5 Contracts) as well as 10:38:15 for (2 contracts).

I'll report the server to NetOps and get the links to QA so they can see the duplicates as well.

Thanks for all your help, hopefully we can get this taken care of once and for all.