Skipping Ticks in Busy Times

[mbuta]

Can someone point me to some examples [if available] of skipping ticks in busy times?
Anything I tried does not work properly and still get "frozen" in very busy times.

Thank you.

[mbuta]

Please ignore this post.
I may have found the solution to my [performance] issue.
Need to test in real time. If still need help I will re-post.
Thank you.

[FibbGann]

mbuta,

This is common for all data providers even if you have a top of the line PC/Server and Inet connection.

If the market experiences high activity due to news or other circumstance many data providers have a lag for many reasons I assume.

When I have encountered these issues though, Esignal normally came into line with the correct data before any others did.

This is something to get used to over time for its been like this for awhile.

I think with current technology there is not much than can be done to correct this.

Its kind of like screaming fire in a small room packed with people. Everyone runs for the exit. Your bound to get a bottle neck of people initially.

[mbuta]

FinnGann,

You may be right in what you say, but in the same busy times, my IB quotes work fine, while the eSignal screen shows the "clepsidra" [CPU crunching].
I detect those busy times and order the efs to do [almost] nothing [for a second] and return previous values. But this is still not good enough, because Data() object [THE ONLY WAY TO GET RFEAL TIME INTRA-BAR] is also very time consuming.

Without this filter, many times, the only to get out is to reboot the computer. I find this unacceptable.

The end result is that we get "blind" in these crucial moments, which may cost us a lot of money and, I think, eSignal should do its best to provide us with an efficient solution.
I had a few suggestions, but I was asked to submit an "official request", which I gace up on, after over 30min of data to fill.

Regards,
Mihai Buta

[buhrmaster]

I, too, have been having problems of late with eSignal's app not keeping up with ticks (I think the problem is with winsig.exe). This now happens nearly every open, whereas in the past the problem only occured on fed days at 2:15. Since I have global new high/low alerts I can hear the new ticks arriving but the screen doesn't update until the market slows. Meanwhile the IB app continues to display timely quotes. My cpu does not max out during this and system memory seems fine. I can also see winros.exe receives the quotes on a timely basis if I use DDE with excel, in which case esignal's DDE quotes keep pace with IB.

My graphics setup is an ATI 9800 with 128 mb, which I would think would be sufficient to handle the graphics updating.

I had been using basically the same page setup for over a year with no problems and have only recently been cutting back on efs usage, trying to solve the slow charts. Nothing has improved the situation yet.

[Guest]

Although I'm new to eSignal. I too have noticed just this past week, eSignal has been rather slow. And I saw that winsig.exe was taking up quite a bit of the system resources. Not sure what it does though.

Are you saying that it has to do with graphic capabilities?

[Guest]

Hi,

FWIW, there is a big difference between how eSignal handles Market Data and Interactive Brokers handles market Data.

eSignal provides all available data (based on how you have set up your chart). This makes it suitable to monitor Volume and other data where you require a complete picture of market activity.

IB provides a snapshot every 300 miliseconds. Here is a description I snagged from an IB API users group that describes the IB feed better than I could...

... IB doesn't report every trade, for any instrument in any exchange. The IB datafeed is optimized to ensure that it keeps up with the market no matter how busy the market is.

To accomplish this, it effectively sends a price snapshot for each instrument at regular intervals. This interval seems to be about 300 milliseconds. For each of bid, ask, and last it compares the current price and size with the values at the last snapshot. If the price is different it sends both price and size. If the price is the same, but the size is different it sends only the size. If both price and size are the same, it doesn't send either. If there have been any trades since the last snapshot, it sends the (accumulated) volume (so where the price and size haven't changed but there have been one or more trades, this can be detected from the increased volume).

By doing this, IB knows the maximum bandwidth required for each ticker, and hence for each customer (since the number of tickers is limited), and so it can size its servers to be able to cope with that load. If a market becomes very busy, it makes no difference because it will still only send an update three times a second or thereabouts, even if there have been 100 trades during that second. This avoids the problem that every other data feed seems to have, where the data will sometimes lag way behind the market at busy
times (with every other vendor I've used, I've had occasions where the data could be anything up to two or three minutes behind the market).

There is an irritating side effect of this technique, which is that price movements between snapshots may not be reported at all: for example if the last price at snapshot 1 is 100, and then price moves up to 102 and then back to 101 by snapshot 2, the price reported at snapshot 2 will be 101, and the 102 price will not be reported at all. This leads to occasional incorrect highs and lows of bars, but rarely by more than one tick: whether that is significant depends very much on the trading strategy used.

The above isn't a complete description, but it covers the basic mechanism. In answer to your question, there are no timestamps in the IB data stream.

[Guest]

A followup to my last post ... this is just general information and does not pertain to Mihai's original question.

There are many issues such as a large number of drawn lines on the chart, re-drawing them every tick, and output to the Formula output window that can affect performance as well.


Here are the links:

http://forum.esignalcentral.com/showthread.php?s=&postid=51315&highlight=slow%2A#post51315

http://kb.esignalcentral.com/article.asp?article=1639&p=1

http://kb.esignalcentral.com/article.asp?article=1301&p=1

[buhrmaster]

Originally posted by wwong112
Although I'm new to eSignal. I too have noticed just this past week, eSignal has been rather slow. And I saw that winsig.exe was taking up quite a bit of the system resources. Not sure what it does though.

Are you saying that it has to do with graphic capabilities?

That is just a guess. I can see that bandwidth is not an issue for me as I can see that excel dde quotes sourced from esignal are perfectly in sync with IB. I can hear that winsig is processing those quotes as I have global alerts with audio feedback and while the market is active making new highs or lows as on the open I can hear the audio feedback but the screen does not update. I have been running the same pages for as long as 18 months but never had the screen not update on the open until recently. FWIW my cpu is not maxed out (unless I'm running excel, which is about the worst cpu hog out there).

[mbuta]

Well,

I did not expect such a vivid reaction, but I am glad it happened. This means that we [eSignal] may have a problem that needs to be addressed and I hope some competent specialist from eSignal will clarify the issue.

As for some comments made here:
a/ I think, what IB does "do whatever to keep up with markets, no matter how buys", is what eSignal should do to.
b/ I know performance is affected by number of plots, coloring, etc., but I am talking about efs with average execution time below 0.5ms

Thank you everybody and I hope we will hear from eSignal in this matter.

[FibbGann]

mbuta,

There have been multiple threads on this over the years. I would suggest searching the forum for further info.

[Guest]

How can we track how much execution time it takes in efs, or a part of an efs code?

That would be helpful for me to profile where the bottleneck is, especially I have DLL call to IB to execute orders.

William

[mbuta]

wwong,

Yes, you can: go to "Tools" -> "Performance Monitor".

[buhrmaster]

Originally posted by mbuta


As for some comments made here:
a/ I think, what IB does "do whatever to keep up with markets, no matter how buys", is what eSignal should do to.
b/ I know performance is affected by number of plots, coloring, etc., but I am talking about efs with average execution time below 0.5ms

Thank you everybody and I hope we will hear from eSignal in this matter.

Mbuta-

I really disagree with your point A. First, without knowing what causes the screen to freeze I wouldn't want to prescribe any course of action. Second, that eSignal carries every tick is very valuable and is one thing that distinguishes them from IB's data feed. If I wanted a snapshot data feed I would drop eSignal and use just IB.

Right now, I'd like to understand the tradeoffs with using different types of studies, charts, alerts, etc.

[mbuta]

Buhrmaster,

Everybody is entitled to an opinion. Obviously your problems are diferent than mine. I wish you never be in the position to have a lot of money on the line and to have to take "blind" decision because eSignal shows the clepsidra. Sometimes I got out ok, but many times it has been very costly!

All I said here is that, obviously, it is an important problem for very many and maybe it is time for eSignal to provide a solution which, in my case, woul;d be a simple way to get the real time c lock [without having to declare Date() object every tick]. I can take care of the rest.

Ability to skip ticks in busy times would be a great feature and solve a lot of other problems [including efsInternal performance issue which, personally, I would love to use use it but is completelly inpractical].