Performance concerns with "getSeries"

[mbuta]

Below is a performance report of the same two stochD studies [14/3/3 and 28/6/6] in three diferent setups:
TEST = EFS1
TESTa = EFS2 plotted with getSeries
TESTb = EFS2 plotted with getValue



As we can see, TESTa [EFS2 plotted with getSeries] is much slower [5-6 times slower] than the other two.
Is this normal?

Note: This is very important, because getValue cannot be used [in a meaningfull way] in real time and, apparently, getSeries is not appropriate for historical values.

Thank you.

[ACM]

Mihai
That sounds about right. Note however that you are comparing very different sets of functionality. When using getSeries() the engine maintains complete syncronization for you at all times and the performance will not vary by much (if at all) when you add external intervals (or symbols). If on the other hand you were to implement true multiple intervals using EFS1 (not pseudo-multiple intervals but true syncronization across all time frames) then you would end up with much worse performance with the EFS1 studies. I have several EFS1 studies written for multiple intervals and their performance does not even come close to that of the EFS2 equivalents.
If you don't need to plot multiple symbols/intervals then use getValue() which as you have seen has performance equal to that of the EFS1 studies. Alternatively continue using the pseudo-multiple intervals
Alex

[DionLoy]

I would also add that .03ms and .2ms are negligible in the context of the system timing clock. Any number of events, such as your CPU's cache, could affect a run of the EFS performance counter. I believe Alex noticed awhile ago that if he loaded one EFS first, it would always take 3 times as long as the 2nd one profiled. When he switched the order the other would take 3 times as long.

We may even change the display to read out < 1sec instead of ms timings.

But, I agree with Alex below, getSeries is doing some additional work, and you can't get additional work for free.

However, most of the heavy lifting of EFS2 on a different timeframe or symbol will be shared whether you use getSeries or getValue ie:

1. calculate the bar values on the time frame requested.
2. sync time frame to base time/symbol.

#2 is the heavy part, and is done BEFORE you get to getSeries or getValue.

[mbuta]

Alex, Dion,
Thank you for clarifications.

Two comments:
Dion, I aggree that .2ms is norhing, but if it has to happen several times a minute [forget on every tick], in 4 windows, each window having 5-6 panes, it adds up to unacceptable. In busy markets everything freezes until it cools down and all you can do [in in a trade] pray it goes for you!

Second comment is about my "pseudo-time-intervals". Yes, they are pseudo, but give me reliable values every bar.
For MA type of indicators [MA, MACD, Bollinger Bands] they meet the EFS2 indicators perfectly at external interval boundaries.


Hoewver, EFS2 allows a lot of flexibility that I wish I could use, specially for generating series out of my functions, but can't overcome the performance issue, yet. I keep trying.

Best regards,
Mihai

[DionLoy]

Hi Mihai,

If you have 10 ticks per second per symbol, in 4 chart windows, with 6 panes (that's 24 EFSs running for a total of 240 updates/ticks per second). And if each of those EFSs are running at .2ms per update, that's a total of 48 milliseconds of CPU time.

Which leaves 952 ms unused. The EFS itself would just be contributing 5% to the CPU usage.

Not to say that you're not experiencing slow downs, but just a note that the EFS performance numbers should be taken with a grain of salt. Treat them for isolating EFS that are on an order of magnitude slower than expected, rather than the actual ms values.

[mbuta]

Hi Dion,

I understand what you are saying, but the reality is that there is a lot of overhead associated with general setup and the fastest I can plot those StochD, for example, is .45-.5ms.
Also, any combination that excceds 2.5ms per tick becomes unusable in busy markets.
Markets like QQQQ and SMH have thousands of ticks per second.

That is why I keep asking how tominimize the overhead. I am around that number, when all the functions are to be executed [not simple plots of stoch, CCI, ADX, etc.].

So, any hints in reducing execution time are welcome.

Thank you

[DionLoy]

Probably the best way is to post some code for others to review, see if there are any obvious logic arrangements that were unnecessary perhaps?

We have been profiling eSignal for each release, the obvious optimizations have been done. There is one other thing that I can attempt with the EFS Engine. If it works, it will make it faster, but at the sacrifice of using more memory. This will be attempted for the next version.

[mbuta]

Dion,

Thank you for the offer, but my script is still a mess, cant expect anybody to make sense of it. Right now, it is manageable, but I still have to add stuff, lots of stuff and then I will be cooked!
I hope to be able to use only one pane and the price study, in the end.

I have a diferent question I wanted to ask: Is it possible in EFS2 to re-use an already declared series inside functions called with efsInternal? Or we need to re-declare in each function [in oreder to be allowed to create a cusrtome series]?

I looked for examples, but everywhere, each series is re-declared.

Thank you,
Mihai

[ACM]

Mihai
Unless I am misunderstanding your question you can pass a series as one of the parameters of efsInternal() or efsExternal() and then use that series in the function or efs.
Alex