Confusing Parameter behavior by esignal.RequestHistory

[lparkh]

Hi,
The interaction between 3 parameters for this function are confusing me: <number of bars>, <start time>, <End TIme>.

My Goal is to get all bars of data between 8am EST and 15:59 EST. Nothing more, nothing less. But I can't seem to achieve this. For example:

If I set <start time> to 480 (my interval is 1 minute) for 8am
<end time> to 959 for 15:59
<number of bars> = 959 - 480 = 480 (adj by 1) for number of minutes between these two times and presumably the consequent number of bars

What I get back is data from 8:30 to 18:04 (not the requested times)!! Both my end time and start time are ignored! I analyzed the results and I think what is going on is the fn is delivering back the number of bars requested at expense of bounds I set. I think is because some bars are missing (no trades on those minutes presumably) so to reach requested 480 bars it has to go past my limits. I could live with this if at least paid attention to my start time and then returned more data then I needed. But as you can see it isntead shifts the time window to achieve it's 480 bars by starting at 8:30!
Since I can't know in advance what bars will be missing I can't assess <number of bars> any more accurately! I tried using 'btDAY parameter instead but that just gives a fixed 8am to 4pm set of data regardless of bounds I set.
How do I get this function to respect the bounds I set on start and end??? Help!!!
Thanks,
Lin

[lparkh]

Even More Distrubing Example!

Suppose I ask for data between 8am and 10am (with 121 bars requested). I get back data from 14:37 (2:37 pm!) to 18:23!!! Looks like it starts from last received bar (it's abou 18:23 right now) and counts backward ignoring my time brackets!!!
Help!!!

[JimBecker17]

How it works is a bit odd, with a significant issue being seems to always need to give you the latest data and work back the requested number of bars. (This kinda prevents one from making multiple requests going further and further back in history.)

So if the number of bars on a given day vary, you will get overlap of days when getting all the bars.

What you might consider doing is set start/stop to -1 like in their example. The data then comes in reversed, but you can choose then to ignore/drop data which isn't in your desired time frame. Request just a bit over the max number of bars you might get.

Yes, it's more than you desire, but at least the data comes in.

I don't represent eSignal but have just had to work through these sorts of issues.

-- ok - guess always need to whine a bit.... -- bad habit..

My big issue at this point continues to be that history requests get dropped quite a bit. It was explained that the history servers prioritize requests, but this doesn't explain the fact that they don't complete eventually. Overnight running requests about thirty completed out of over four hundred requested. Looking at the memory behavior of eSignal it seems to just kinda seems to be forgetfull, but can't determine if this is client or server side issue. Requests going out beyond a week of data are the most likely suspects, most newer than that get serviced.

Am hoping one of the other problems fixed will correct this in the next release/beta of 7.7. Either that or others will discover this and multiple whiners might have someone take a look at it. Could be just me, but probably not. (Have seen data requests coming in on handles other than what was requested, so it's probably a minor problem with sorting out data coming in.)

But in general things are working pretty well. This aspect can live with for a while. Had to do a lot more effort to make other data provider software behave! eSignal is quite good comparitively!


-Jim

[Guest]

I found out that the starttime and endtime should actually be labeled DATES, not TIMES, which would explain the problem. The responses start with the most recent bar of data, going backwards until the number of bars have been filled, which is the way our charts load. There is filtering done in the eSignal application so your applications will need to so as well. We would rather send too much data than not enough.

JimBecker, I don't understand why you need to make so many requests. Can't you get say 6 months of historical data for x amount of symbols then just update your database with the new records?

[JimBecker17]

DATES not TIMES! Well that is great to know. It's documented as start and end hours - which seemed a bit odd.


My application is setup to get 6 months data then calculates how much more is needed to update with new records over time, updating symbols that are a week out of date. Good citizen and all.

The problem is only about 10% of the requests longer than a week ever finish, no matter how much time is given for them to complete. Someone drops the ball on processing the data and getting it back to me.

It could not possibly just be your servers are too busy - all this I have been doing in vain in the middle of the night and also during the holiday weekend! Not a great holiday for me!

Considering that the doc is very wrong on the parameters and you guys just uncovered it seems your test programs might not have flushed out this form of behavior. Perhaps the recent test program which gets bulk basic quotes could be expanded to show getting and matching history requests. Think that would turn up the problem for you guys to see. If it comes down to it I'll update it for you if eSignal will look into this problem...

I'll do a long run over Saturday and Sunday nights this weekend and save the log of requests/responses with the times for you guys. Will try to private message it to Robi, rather than post to the forum.

Thanks for any help - trying not to whine but am a bit at witts end here. Plugh!

-Jim

[JimBecker17]

FANTASTIC NEWS! Just after posting my last whine about this problem discovered Robi had taken the initiative to provide us with a "pre-beta beta" of 7.7. Was hoping that the Event Sink crash fix (or perhaps another) would solve the issues which have been plaguing me for months now.

AND MY ISSUE IS FIXED! The history requests are 100% completing, and they are coming in at most around five minutes, which is fantastic. Prior to this was only getting about 10% completed and some taking as long as five hours when they did complete.

So to whomever fixed this problem, THANK YOU VERY MUCH! If I knew where to send a case of MicroBrew would have it off to you on Monday! Am a happy camper here after discovering "it's all good" now with the History Requests.

-Jim

[Guest]

I'm glad I was able to make your day. I've been wanting to get this out for a while, but we had to wait for the eSignal release, which was postponed.

It just so happens that I am fan of micro-brews, so if you truly feel that you need to show some sort of gradification, you can send it to our office under my name and it'll find me - and I'll save some for Simon.

[mserpell]

Robi,

Can you pls post some samples of starttime and endtime, I seem to be having some trobles guess ing the correct formats.

This,

Matt S.

[LTANESIGNAL]

Yes some clarification would be appreciated.

When I check in object browser I still see lstartTime and lendTime declared as int.

I also tried the following fragment in c#

int glhistoryHandle =
esignal.get_RequestHistory(
symbols[idx], "1",
IESignal.barType.btBARS, 10, 38200, 38201 );

where 38201 represents the '04-Aug-05' and found that esignal ignores this and returned data from today.

I 'm using release 7.7 beta 5.

[sthornington]

My advice would be to pass in -1 to those arguments and filter the data in your code, I don't believe they work at the moment. They may work in the future, but I can't tell you when that will be.

Simon.

[mserpell]

If I read this correctly the -1 to -1 paramters are not currently working to deliver trading hours only data, it is up to the client side application to strip out bars in non-market periods.

This has been a major cause of frustration as the documentation suggests it should be working. Also this bulletin board is filled with similar issues and there has not been a clear response.

I would suggest esignal take some corrective action by updating the documentation and advising their customers who have applied for this service of this issue.

BTW : Are there any other undocumented features we should be aware of