Follow up on crash reports?

[jimcole23]

I'm using the Desktop API in eSignal 7.8, build 688. The problem I'm about to describe also happened with 7.7; I think it happens a little less with 7.8.

My app is a .NET program written in C#. My computer is a reasonably fast Pentium 4 with 2GB of RAM. My app streams quotes on a handful of stocks (10-15), and once/minute, requests today's daily bar for about 120 stocks. The Desktop API crashes a lot, several times a day. I usually take the time to send the crash report to eSignal. Judging from the contents of the eSignalCrash.log file, however, not a lot of information gets sent to eSignal. Is there any way to follow up on these crash reports and find out if the eSignal developers are able to narrow down the cause of these crashes?

Is there any way for me to put eSignal into a debug mode, or turn on some extra logging, so that I can try and figure out what's causing these crashes?

By the way, call me naive, but it's my belief that the Desktop API should not crash, no matter what my program does. It should defensively check for invalid parameters, requests on closed handles, etc. I've already written a lot of theoretically-unnecessary code in my app to work around weaknesses in the Desktop API. For instance, I queue up my historical requests and send the next one only when I've received all the data for the previous request, since sending it requests for historical quotes on 100 stocks killed it (yes, I know the teeny little sample app works). I "expire" requests if I don't get data back in a reasonable period. I've read other posts where people mention that they've got watchdog threads monitoring their app for eSignal crashes and hangs and restarting eSignal in those cases. For software and services that I'm paying $1500/year for, I guess I expect it to actually work.

[Guest]

Well, the Desktop API tends to crash when responses to requests are not handled correctly. There are numerous ways to make it crash, just like almost all software applications. There are over 200 users of the API and the vast majority are satisfied, so it really depends on what and how you are using it. As far as what you are paying, only a small fraction is for programmatic data, most is access to real-time market data itself.

Can you post which method you are finding the crashes or send a piece of code that consistently replicates these problems?

[jimcole23]

I can't send you all of my code. But give this a whilrl: Run the Multiple History Requests in VB sample under Visual Studio .NET, resetting the symbol DB in DataManager each time. For me, it sometimes works fine and retrieves data for all 100 symbols. But often, it retrieves data for anywhere from 20-60 symbols at a fairly consistent speed, then does one of three things:
1. Slowly fills in the other symbols over the next few minutes
2. Never fills in the other symbols
3. Crashes

If it succeeds in getting all the data, either quickly or slowly, and I exit the sample program, eSignal often crashes. I've run it about 15 times in a row now, using both the Debug and Release versions, and it's crashed one way or another at least half of these times.

Is there a steep cost in terms of cycles to get events from the ActiveX world into the .NET work via the interop layer? Are you planning to provide a native .NET interprocess communication system to the desktop app soon?

[Guest]

OK. You may be running into a protection mechanism our server farms have preventing a high number of requests from the same user. This mechanism prevents a single user from "hogging" resources by continually asking for data. Simply put, I cannot provide a concrete explanation because it only happens occasionally.

We're not examining the possibility of providing .NET interprocess communications at this time, mainly because not all users will install the .NET platform on their desktops. eSignal is compatible with .NET, but does not run on it.

[jimcole23]

I'm not sure I buy that explanation. Do other people get this behavior running the Multiple History Requests sample out of the box? Why would you ship a sample that is going to trigger that response from your server farm? And why would the server farm throttle cause crashes on my client, especially when all the requests were successful?

My app isn't requesting that many symbols. I store previous days' daily history in Oracle, so when my app starts up, it basically starts requesting daily history on ~120 stocks once/minute. It also sends the requests one at a time, not all at once. If there was an easier way to get the last price AND the total volume, I wouldn't request daily history at all.

If my subscription is supposed to be for 200 symbols, I would think I'd be able to request current the current value of up to 200 symbols at a reasonable rate (e.g., once/minute). I'm only scanning ~120 symbols. Not 120 at a time; 120 total.

I suspect the ActiveX-to-.NET path. I'm guessing that these samples work great if you compile them with some pre-.NET version of Visual Basic; I think I still have VB6 on another machine, so I can try them from there. I would also assume that if one of the main samples didn't work for most users, you would have heard more outcry.

Have you folks done much testing of desktop clients running under .NET? If this is the cause of these problems, then I'm not sure what to do other than:
1. Rewrite my code so that the handlers do nothing other than store a copy of the data, so they run as fast as possible. Right now, they're doing some very lightweight alert checking as well (e.g., has price gone up $0.10?).

2. Wait for eSignal to provide a native .NET interface in addition to the ActiveX version.

[Guest]

It sounded like you were running the sample numerous times - "resetting the symbol DB each time". The sample works fine, so it seems there's something else that you are doing that is making the API choke and there have been many people using .NET.

Are you requesting daily history for 120 symbols once every minute? Rather than requesting each minute, which is where you are running into problems, you should stream data. Couldn't you use T&S updates to keep track of intra-day volume and current price?

[jimcole23]

I don't see how T&S helps with volume. It gives me the lot size of each trade, but I don't see where it gives me the total volume for the day so far. I could, of course, iterate over all the T&S bars and sum up the volume, but that doesn't seem easier than just getting the daily bar, which gives me the volume already summed up. Am I missing something about the T&S data?

[Guest]

From what I can tell, you are in search of intraday volume and I am suggesting that you use streaming T&S data to overcome the issue with using daily data. You should not be making historical requests every minute and we have controls in place on the server farm to prevent users from making such timed requests, which is what I believe you are running into. I am just giving you explanations for the problems you are facing with your current implementation. Even in eSignal, intraday volume is calculated on the client side going forward after the initial request.

[jimcole23]

Yes, I need accurate intra-day volume for all the stocks I'm following. It's a crucial variable in my algorithm.

So before I rewrite a whole bunch of code, please confirm for me that eSignal would rather have me streaming T&S on 120 stocks so I can keep track of volume, than requesting today's daily bar once/minute. It seems like 120 T&S streams will end up sending a lot more data to me than the periodic historical requests, especially if my program starts up partway through the trading day. In that case, I have to retrieve every single T&S tick right away for all 120 stocks so I can sum up the volume to that point. That's hundreds or thousands of ticks * 120 stocks. To me, that seems way more costly, for both eSignal and me, than the first 120 historical requests my program issues when it starts up.

If my program is started after the market opens, I'm still going to have to ask for today's daily bars for all stocks once, to get an accurate value for open. Since the T&S stream arrives in reverse order (most recent ticks first), and you can't tell when it's done, there would not seem to be a 100% accurate method to calculate the opening price from the T&S stream. If you don't mind your value for open(0) changing after you've set it, then you could get it from the T&S stream. But if you're calculating and immediately making decisions based on the gap between open(0) and close(-1), then having open(0) change is not good. Watching the gap is also a critical part of my algorithm.

It would be good if you could confirm that my requests are indeed being throttled by your server farm. I assume you have logging there that you could look at and see if my account has been throttled at various times. If I'm not being throttled, then what is the explanation for the data I'm not getting?

Also, none of this explains the eSignal crashes, especially the ones I saw with the eSignal-supplied sample application.

Lastly, what are the limits or rules on requesting historical data? I'm going to add back-testing to my program, and I'll be requesting 3-minute bars for one stock at a time, as far back as I can go (6 months, right?). Then I'm going to request that data for the next stock, and the next one, etc. When that test is done, my program will change one parameter and do it all over again. And so on. This doesn't seem any different to me than what eSignal does if you back-test EFS code, other than being more automated. I'm not going to do this type of intensive backtesting forever, but I need to figure out the right value for a dozen or so parameters.

[GenSpoo]

Originally posted by jimcole23
Since the T&S stream arrives in reverse order (most recent ticks first), and you can't tell when it's done, there would not seem to be a 100% accurate method to calculate the opening price from the T&S stream.

You can tell when all the T&S have loaded. I believe it is called IsTimeSalesReady(handle), but I don't have it in front of me at the moment.

Cheers... George

[jimcole23]

The behavior I see using the VB Time & Sales sample app is that OnTimeSalesChanged is often called multiple times after you make a T&S request. Each time it is called, IsTimeSalesReady(lHandle) will return true, and you get some of the T&S ticks. But a little while later, OnTimeSalesChanged is called again with more ticks, and these ticks occurred earlier in time than the ticks you got on the first call to OnTimeSalesChanged. So that's why I think there's no way to calculate open 100% accurately from T&S.

Suppose you saw a tick with a timestamp of 9:30:15 and decided it was open, because it was the last tick in that call to OnTimeSalesChanged. So you set your version of open(0) to that tick and make some decisions based on that value. But along comes another call to OnTimeSalesChanged with a tick from 9:30:07, and the price is different enough that it affects the decision you already made.

I've noticed that if you immediately request T&S again on that symbol, the data seems to arrive all at once, presumably because it's in eSignal's cache. So you could calculated open(0) accurately by requesting T&S twice. But I don't want to depend on that type of behavior in order to make my program work. Many events out of my control could affect that cache (i.e., it's not my code, and the behavior isn't documented or guaranteed by a public API), so I can't assume that it will always behave the way I happen to see the sample app behaving.

[jimcole23]

Ok, I've tried getting volume from T&S data. If I pull all T&S trade data for ADBE from 2/11/05 and sum up the lSize values, I get 2,266,246, including trades before 9:30 and after 4:00. But Yahoo Finance, eSignal, and every other service I try reports Friday's ADBE volume as 2,452,735. If I only count trades from 0930-1600, the total volume from T&S is 2,208,513. That's about a 9% difference. Why the huge discrepency?

Several stocks with lower volume that I tried sum up to the correct value. For a more-heavily traded stock, AMD, the reported volume Friday is 9,377,300, but the T&S sum is 9,393,700, a difference of 6,100 shares. For DRIV, it's 2,886,949 vs. 2,701,388 from T&S (2,858,934 if I count pre- and post-market activity).

Unless I'm doing something wrong, it appears that T&S data does not give reliable volume information for moderately-traded stocks.

I would be very interested to know what Robi for total ADBE volume on Friday by summing up T&S data.

[Guest]

I have asked our QA staff about this and the only reasonable explanation is that you do not receive trade cancellations, insertions and deletions (CIDs) via T&S, whereas the total volume for the day reflects it. This would explain the 6,100 difference in AMD, but may not explain ADBE. Other trade conditions, such as rule 127 block trades, which are large-blocked trades agreed outside of the general market, may explain the 9% in ADBE. We are working on supporting CIDs and other abnormal trade conditions in our feed and should be a part of it in the next few months. It is currently in QA.

[jimcole23]

So, Robi, what I'm hearing you say is:

1. T&S data is incomplete, and so I cannot figure out the day's volume using T&S data. You're working on addressing this issue at least partly in a future release, although you're really just guessing at why the T&S volume for ADBE on 2/11/05 was 9% below the actual volume.

2. I'm not supposed to, and will not be allowed to, request today's daily bar once/minute in order to get accurate intra-day volume.

If I'm hearing you correctly, then:
1. There is no supported way to get accurate intra-day volume via the Desktop API in eSignal.

2. T&S data may be extremely suspect in general (i.e., you don't know why almost 10% of the transactions are missing for ADBE, 6.5% for DRIV, etc.)

Is this the official word from eSignal; i.e., "it's broken, we don't know why, and we won't let you use the part of the API that works."?

I'm hoping to get a definitive answer here. You can tell me that most of the money I'm paying every month goes not to eSignal, but rather to the feed providers, so I shouldn't say "it's expensive software, so it should work." But the cost is not the point. The software I've written that uses this data is used to make rather large financial decisions, so it's important that the data is accurate. Not close, but 100% accurate.

I would love it if you were able to demonstrate to me that you got accurate volume data from the T&S stream for ADBE and DRIV on 2/11, and that there's something wrong with my code. But my code that sums up the T&S data is pretty simple, and you haven't demonstrated that it's possible to get the correct value for volume from T&S.

Is eSignal an accurate tool that traders can really use, or is it a toy? If it's not a toy, then I think all eSignal users need to know why T&S data appears to be off by almost 10% in some cases. If it is a toy, then I need to find a new provider.

Do I need to do something more official in terms of reporting what appears to be an extremely serious bug, or is this forum the best place?

[Guest]

I did not track ADBEs volume on 2/11, so I don't know what the intra-day sum could be by tracking T&S. I am letting you know that CIDs are not sent in the feed in real-time, but are applied to the volume for historical quotes. I also mentioned that there are mechanisms on servers to prevent single users from tying up servers with1 minute timed requests, which is likely your original issue. I am giving you what I know so you can modify your application to fit your requirements. I am also stating that support for CIDs is currently underway, in testing and will be a part of the feed in a few months. eSignal Pro software application is used by some 15,000 professionals which do not consider it a toy.

We also have another API, the Standard API, which is more expensive and geared for commercial development. You may be interested in that. If you are, please fill out this form.