Follow up on crash reports?

[jimcole23]

How can I modify my application using the information you've provided when you're giving me two choices to obtain the data I need: One method that gives wildly inaccurate data, and another that isn't allowed?

You said that the eSignal application itself keeps track of intra-day volume by summing up T&S ticks:

Even in eSignal, intraday volume is calculated on the client side going forward after the initial request.

If you're right, then its data was off by this much during trading hours today on these symbols:

2/14/2005 ADBE T&S volume: 1416685, Real volume: 1593384 Difference: 11.09%

2/14/2005 MSFT T&S volume: 56250561, Real volume: 58933740 Difference: 4.55%

2/14/2005 AMD T&S volume: 4852300, Real volume: 4921000 Difference: 1.40%

2/14/2005 DRIV T&S volume: 1591222, Real volume: 1721582 Difference: 7.57%

2/14/2005 INTC T&S volume: 56947582, Real volume: 59214780 Difference: 3.83%

[jimcole23]

Since the market is open, and we're well into the trading day, I just checked ADBE in eSignal, and its volume appears to be correct. I don't think Robi was right that eSignal sums up T&S ticks to get intra-day volume. If it did, the volume eSignal displays for ADBE would be far lower than the correct value, given that eSignal is leaving out about 10% of ADBE transactions from its T&S stream.

[Guest]

I wrote, as you quoted me, "Even in eSignal, intraday volume is calculated on the client side going forward after the initial request.".

I did NOT write, "the eSignal application itself keeps track of intra-day volume by summing up T&S ticks". There is a significant difference between those two statements.

You never mentioned which function you were calling to get intraday quotes, but an engineer informed me that you should use "XMLGetBasicQuote", rather than "GetBasicQuote". You should try getting streaming data with that function rather than T&S and you may get the results you are looking for.

[jimcole23]

So you're suggesting I stream quotes, not just trades, for 120 stocks, to keep track of intraday volume? That's quite a lot more data than even the T&S stream.

I can't find any documentation on what you values you send in to the XMLGetBasicQuote method, or what data you get back. Can you please point me toward it?

[JimBecker17]

Re: Follow up on crash reports?

For what it's worth, i kinda gave up on the crashes really getting fixed. But here I am again on a Monday with a hundred+ crashes so thought would just spend some time seeing if anyone else had taken up the bandwagon.

it's impossible to get work done with eSignal just crashing.. I've put in logic to capture and categorize their crash info, but apparently it tells them nothing. have also volunteered to do any debugging possible to give them call stacks or other...

Thank you Mr. Cole for trying. Hopefully you aren't totally worn out yet. The reported problems with the 0x014 crashes and the crashes on exit have seemed better, but aren't fixed. I still contend they probably are triggered by bad data, as they show up in clusters rather than repeatable.

The change in behavior that did occur was long periods of eSignal stopping doing anything, even updating it's clock. I don't know if it's looping waiting for something, but it will just freeze up.

I put a shortcut hotkey in my app to kill and restart eSignal. "K".

Echo full agreement that crashes are unacceptable in supported software. Can't imagine Boeing saying their new plane is cheaper but the side effect is it crashes more.... Ha!

-Jim Becker

Originally posted by jimcole23
I'm using the Desktop API in eSignal 7.8, build 688. The problem I'm about to describe also happened with 7.7; I think it happens a little less with 7.8.

My app is a .NET program written in C#. My computer is a reasonably fast Pentium 4 with 2GB of RAM. My app streams quotes on a handful of stocks (10-15), and once/minute, requests today's daily bar for about 120 stocks. The Desktop API crashes a lot, several times a day. I usually take the time to send the crash report to eSignal. Judging from the contents of the eSignalCrash.log file, however, not a lot of information gets sent to eSignal. Is there any way to follow up on these crash reports and find out if the eSignal developers are able to narrow down the cause of these crashes?

Is there any way for me to put eSignal into a debug mode, or turn on some extra logging, so that I can try and figure out what's causing these crashes?

By the way, call me naive, but it's my belief that the Desktop API should not crash, no matter what my program does. It should defensively check for invalid parameters, requests on closed handles, etc. I've already written a lot of theoretically-unnecessary code in my app to work around weaknesses in the Desktop API. For instance, I queue up my historical requests and send the next one only when I've received all the data for the previous request, since sending it requests for historical quotes on 100 stocks killed it (yes, I know the teeny little sample app works). I "expire" requests if I don't get data back in a reasonable period. I've read other posts where people mention that they've got watchdog threads monitoring their app for eSignal crashes and hangs and restarting eSignal in those cases. For software and services that I'm paying $1500/year for, I guess I expect it to actually work.

[GenSpoo]

You've had over one hundred crashes of the eSignal app just today alone? I'm curious what sort of stuff your app is doing (number of symbols, timeframes, t&s or bars, etc.). I don't recall having a crash at all with my app running (not saying you aren't experiencing it).

Cheers... George

[jimcole23]

I've eliminated virtually all of the crashes of the eSignal ActiveX control while my app is running. It usually still crashes when my app exits.

I've done this by being very careful about how I call the eSignal API. For instance, I don't bombard it with requests all at once; I queue my requests up and submit a new one only when:
1. The previous request has completed, or
2. Two seconds have passed

I've also reworked my code so that it does the minimum possible amount of work in the event handlers. My handlers basically copy the data from eSignal, and code that runs on a timer comes in every few seconds and distributes as much of that data as it can during its timeslice. The data distribution code sometimes falls a little behind, but it usually catches up with the incoming data pretty quickly.

Again, I feel strongly that the ActiveX control should not crash even if the client (my app) calls it in some invalid way (e.g., calling for data on a closed handle). Obviously, my code shouldn't do invalid things, but if it does, eSignal's behavior should not be to crash. It should protect itself against that, and it does not appear to have even rudimentary protections in place. It's an extremely fragile API. And the responses on this thread from eSignal people have basically been "we have 15,000 customers for eSignal; therefore, we must be doing a good job," which doesn't give me a lot of hope for these problems getting addressed.

[JimBecker17]

If you go back through all my posts over time you can see a progression of more and more logic to handle eSignal problems. This goes back to RequestHistory not working correctly as well as the T&S data being flawed - misordered and missing. Have asked for an array for this but guess was dreaming... (GenSpoo asked for this one i think Nov 2003 - might take a couple hours to implement...)

My logic includes having lots of involved internally queued requests to eSignal, staying under a certain threshold, as well as handling both messages and polling to make sure get data. Then everything that i get is cooked and stored in my own formats, so i don't have to ask a second time..

Of course also have threads that watch for crashes and also for the CoreReport application to show up - so i can shut these down and restart eSignal.

Basically have spent way too much time trying to resolve issues which should not exist at all. The other people who have reported these type issues are just using other products now.

Most days eSignal works with just a few crashes, but some days (particularly Mondays) there are continual crashes. Today it seemed everything was running for about an hour but then i noticed that it was 9:30 and the eSignal clock said 8:28. Ha!

Answering GenSpoo my app is doing DDE quote updates and then timed five minute period RequestHistory calls. The RequestHistory splurts out requests every few minutes as it works its way through groups of stocks so it doesn't give eSignal too much. Normally gated at 40 open requests, which i process at a slower rate than eSignal does.

But i can have my app basically doing nothing, and eSignal still pops up crashes on certain days.

Another app does T&S data, but i don't even try to run that during market open hours. i also don't try to get quote data anymore either, as single entry requests take forever. no array.

Note that just about everything i've done has shown up problems in the ActiveX world which have been a "fight" with eSignal to recognize and rectify. Go ahead and do a search on my name to find out all these details. Have spent more nights/weekends on issues than care to recall...

I keep around the crash logs, but most of them just have some ok status stuff followed by "Crash at <time> - <address>", and the address varies.

By the beginning of this year just resolved would live with it, but today decided to checkin and saw that Jim Cole had taken over the torch. Thanks Jim!

-Jim

[DionLoy]

Hi Jim,

The last time I checked up on some crash reports from you I mentioned that they unfortunately did not contain the information necessary for me to find the location of the crash.

I really do need the EXE that you run in order to duplicate the crashes on my side in order to debug it properly. I know you've mentioned in the past that you are unwilling to provide me this EXE due to some large data sets that it requires. However, unless there is a smaller EXE that you could provide, I'm not sure what else we can do on our side to track these crashes down for you.

Please please please, send me an EXE that you are using, as well as any dependencies, so that we can look at this for you. The crash reports just don't reveal enough in this case for me to be able to find your problem.

Thanks!!

[JimBecker17]

Hi Dion,

Thanks for the reply - I know you are a busy guy. Also sorry about the tone of my postings, get a bit irked with this problem... And it's Monday!

You are correct in recounting my inability to provide EXE and data for complete picture. At one point did write a little program which showed the problem for you, and this had you find at least one instance of the 0x014 crash. But for numerous reasons the complete environment cannot be provided external to the company. Can email details if needed.

As an option I have offered to run a debug eSignal and give you stack traces and variables if possible. Don't mind NDA if needed. If eSignal can't do this another option is improving the Crashlog file so it tells you a bit more info? Over time I'm sure have tried to send over 50 of the CoreReport files, but unclear if that gets you anywhere as well.

If you read between the lines of what myself and Jim Cole have experienced, we are both trying to head off problems by spoon feeding requests. Perhaps augmenting my test app to ask for more RequestHistory() data would cause the problem to arise once again. Chris James and I at one point were trying to compare time of day crash results, because it certainly does flake out in clusters.

My hunch is it's actually ReleaseHistory() in my case letting something go before it's time. Disabling it at one point seemed to help.

In a general sense building up your end so the Crashlog and Corereport files give you "the whole enchilada" will head off the next whiner that comes along after me as well.... I may disappear into the weeds, but the next guy will take over...

-Jim

[DionLoy]

Hi Jim,

Thanks for sending me that sample app. It was definitely helpful in finding the location of some of these problems. I'm tracking down one last item, but I hope to include these fixes for the Beta/3 release.

[JimBecker17]

If this problem is finally nailed I will be very happy. It happens inconsistently, but when it does comes down on me like a hailstorm. Nothing like having eSignal crashing like a tommy gun going off to blow my trading focus...

The test program caused it periodically, the real program runs a lot more stuff and gums up the works pretty good.

Thanks for looking into it prior to Beta, Dion. Sure hope Robi treated you to some beers.



-Jim

[DionLoy]

BTW; just an observation, the IHooks was meant to be instantiated only once per application.

[DionLoy]

Ok Jim, I've fixed the last problem, which was a memory corruption.

Your test app still hits winsig a little too hard. Eventually the COM failure is 'System Call Failed". This is Windows stepping in and stopping the COM call from going through because winsig is busy handling other COM requests.

However, winsig should no longer be crashing on that error (at least with this test app).

[JimBecker17]

Thank you for the efforts, Dion. Appreciate your putting attention on it.

I'll be very happy if the 0x014 error is gone, most certainly.

My normal app has just one IHooks and such, but there is a lot going on as well. It does DDE requests to WinRos, which could also plug up WinSig is my guess. I'll keep an eye out for the System Call Failed as well.

If you would like me to try the build with my app let me know when the beta is up on a server. Thanks!

-Jim