EFS2 Custom Data Series - Perfomance Issues

[ACM]

Mihai
Unlike EFS1 studies which have a relatively straightforward syntax/construct EFS2 studies can be written in many ways some more efficient than others. Posting images of the plots does not provide any indication as to how you have written these studies and only serves to further this discussion in an unproductive way.
If you care - as you say - to make your life easier then post a working representative example of the code that is returning these high performance readings and also the equivalent EFS1 code so that others can compare and (hopefully) help out.
Alex

[DionLoy]

Mihai, unless you post your code, we are unable to help you determine why you're seeing this.

Of course, if you were using the intervalFactor in EFS1 to simulate multiple time frame analysis, and using a real multiple time frame synchronization in EFS2, that's your problem right there. You're comparing apples to oranges, and of course if you pull down the actual multiple time frame data the data calculation times will be more. If you're more than happy with the EFS1 estimates of multiple time frame values by all means just stick with it. That only has to calculate values on one time frame, but estimate the values on the other. It is impossible to expect to perform the actual calculations on the multiple time frames (and pull that data down from the servers) in the same time as your estimated values on one time frame.

Repeat: From what I've seen from your messages, your EFS2 and EFS1 code is not functionally equivalent. You are pulling down more data (multiple time frames) and doing more calculations than in your EFS1 code, which is simply doing an estimate on those other time frames.

I'm afraid we can not investigate this further unless you can provide code that shows otherwise.

Thanks.

[mbuta]

Dion,

You are absolutelly right about "pulling down three data streams in EFS2" vs one in EFS1 and this is why I declare my sources first and use them in other indicators. Furthermore, I eliminated one layer of processing in EFS2 [I use the hlc3 directly instead of its sma(3)], but I expected this to be the only additional overhead.

I thought that, once generated, a data series resides in a buffer in my computer, not in a remote server and that studies that use those data series as source will also look in my computer for data.
Apparently, I was wrong about that.

Happy with EFS1 emulation of multiple timeframes? Well, almost and the not 100% accurate results don't bother me that much, but the inability to re-address that calculated data is what made me convert to EFS2, where this can be done so much easier.

Alex,
Really, I do not intend to generate fruitless discussion, but to clarify issues that [based on the number of people that look at this discussion] seem to be of interest to others as well.

I think Dion caught exactly the way I am doing the studies and pointed out the nature of my performance problems, but I will try to post a stripped down version of my script in the future.

Thank you for clarifying the issue of higher time-intervals. This will allow me to reduce even more the execution time, by updating M1/M2 varibles only on new bars at those intervals.

Note: Now I understand why the EFS2 based indicators do not plot correctly in real time and have to "reloadEFS" once in a while [setBar does not help for higher intervals].

Thank you,
Mihai

[ACM]

Mihai

Note: Now I understand why the EFS2 based indicators do not plot correctly in real time and have to "reloadEFS" once in a while [setBar does not help for higher intervals].

You actually don't need to use reloadEFS() at all unless you are plotting the value rather than the series.
If you run any of the studies that are available in the EFS2 Custom folder and you set them to use an external interval then you will see that they maintain full syncronization with the external interval even in real time and no reload is required.
Alex

[DionLoy]

Hi Mihai,

Once pulled down from the server it does reside in your local computer buffer. However, if you pull down a 1 minute stream, and a 5 minute stream, the 5 minute stream will be a separate stream than the 1 minute one. In a later version this year we will be introducing a local data cache, which will cache all 1 minute and tick data locally, and build other intervals from that cached data.

However, another point to the equation is that even if the 5 minute was built from the 1 minute, in your EFS2 scripts, you are running the studies on the 5 minute as well as the 1 minute, which takes twice as much computational time. In your EFS1 with intervalFactor, you are just doing the calculation once.

To clarify Alex's message regarding the reloadEFS() ... There is special handling in the EFS engine that if you return a Series object rather than a number value from main(), it can detect that the Series was from a higher time frame and properly adjust the return values in realtime. Otherwise, if just returning a number value that was earlier calculated from a higher interval, the engine has no way of determining the origin of that number.

I'm glad we've got all the issues straightened out, I do hope that you'll be able to find a comfortable compromise in your scripts between the older behavior and newer functionality.

Thanks.

[mbuta]

Alex,

I know exactly what you refer to, because this is one of the reasons I got into performance trouble.

I never use raw bultin data series, I process [normalize] them before using, so I cannot plot my data series [correctly] unless I declare them with efsInternal.
You know the rest of the story.
For reference, I published a stripped down version of my scrip in Multi Time Intervals File share. Note that there is no processing [normanilzation] applied there, but you can see that I am respect the principles, as recommended by you and all other competent users. When processing is applied, the fun starts.

An other reason I chose [in EFS1] to return the values is that on busy markets, it is impractical to update complex plots every tick. Even like this, I get frozen for periods of time, exactly when I wish I knew what was going on more than ever [I am sure you understand what I am refering to].

Thanks again,
Mihai