History Server response inconsistency.

[JimBecker17]

Using latest Beta of 7.7.

Am still trying various ways to consistently get historical quote data (five minute). Originally got this code working two+ months ago, but have tried various reworks to tickle the servers without consistent results.

My means now includes both message handling and periodic polling for completion for queries sent. (IsHistoryReady() used for polling).

Most requests have been making are about three months of data.

The situation which continues to persist is very inconsistent response behavior from the servers. Much of the time there is no response, but sometimes am getting continuing active responses.

For a brief moment it was giving me quick turn around time and lots of data, but now nothing. This quick turn started perhaps a couple hours after requests had been made, of all things!

For similar requests and different symbols, have gotten responses ranging from about 15 seconds to as long as 56 MINUTES before they complete...

However most of the time not getting completion of requests, no matter how long I wait.

My current run have thirty five requests active, adding one every thirty seconds. Nothing has happened while I have been sitting here typing, but suddenly the requests starting at around request 32 started showing up... so for twenty minutes nothing, now something. Am getting more now of the newer requests.. All the old ones seem dead... hmmm...

If there is someone there at Server Central that wants to take a peak at what my client is doing, they might clear up mysteries myself and others are seeing...

Some of the new features in 7.7 look great! Still getting some crashes however which have been relayed to you guys via your crash tools.

Thanks for the support! Hope everyone had a nice weekend.

-Jim Becker

[Guest]

I have some information for you.

Since you are requesting such a large amount of data, your requests for data many months ago are being pushed back in the queue. Our servers are configured to fulfill as many requests of the most recent data as possible equally across users. If you are the only one looking for say "F" between 1 and 2 months ago, your request will be fulfilled after many real-time quote requests of highly viewed symbols. So, in essence, the servers are basically responding when they have taken care of more immediate priorities.

I would look at taking a smaller survey of records, say 2 weeks and see if you are getting the same performance. Also I think there should be a significant difference between running your application during market hours vs. after market hours.

I hope this helps. I'm glad you like the new features.

[JimBecker17]

Thanks for the reply, Robi.

Can fully understand why eSignal would set things up like this.

With the priority scheduler mechanism is there any provision for quantums to be provided to requests regardless of popularity?

Meaning if asking for some odd company will the servers give me at least minimal processing so eventually the request will complete? Round robin everything at least once in a while..

I think this is what is cooking me. I get the feeling my requests for obscure companies are pushed so far to the back of the bus they never see the daylight of any cycle time by the servers....

Have seen some complete in around three hours....

FWIW: To get this much data I've mostly been trying to run this accumulation overnight, and once it comes to my system it all persists so will not be requested again. During the day my intent is might have to get a few days/weeks to bring older info up to date. Requests for newer data continue to work well for me, and do show up much faster than anything in the "way back".

Thanks!

-Jim

[Guest]

Your request will complete, but you should carefully make your requests so that you do not repeat anything. That will definitely speed up the retrieval process. Basically, you may struggle to get things backfilled, but after that, you should not have nearly as many issues.

To get minimal processing, you should "break up" your requests into batches and then you should recieve better performance.

[JimBecker17]

Great, thanks Robi.

Yes, tracking everything so repeating shouldn't be the case.

Originally intended to back fill data a month at a time, but it doesn't seem that calendar time can be specified in the RequestHistory() call. Thus the big requests when need to load up a symbol.

Perhaps a future enhancement would allow RequestHistory() with calendar time for either start or end? So could backtrack as needed without a factorial overload of newer data already fetched.

Another enhancement which would be useful is letting me set the Layout used. For me it would be great. Don't know if anyone else cares, but I have weird needs sometimes...

I'll be quiet now... Everything else is working pretty well for me.

-Jim