efsInternal: How to use calculated vars/constants?

[mbuta]

Hi,

Are there any restrictions in using calculated vars/constants in functions used with efsInternal?

If yes, are they explained anywhere?

Here is the problem:

var AvB;
stMA1;
stMA2;
stOsc;

function main() {
if (ALLBARS) {
AvB = ...; //Calculate it, so I can use it in other studies
stMA1 = ...
stMA2 = ...
stOsc12 = efsInternal("MyOscF",stMA1,stMA2); //This does not work
stOsc12 = efsInternal("MyOscF1",stMA1,stMA2,AvB); //This works
}

}

function MyOscF(stMAf,stMAs) {
return (stMAf-stMAs)/AvB;
}
function MyOscF1(stMAf,stMAs,XX) {
return (stMAf-stMAs)/XX;
}

MyOscF return only nulls;
MyOscF1 works fine, but it is not what I want because I may have many other vars I want to be able to use.

Looks like, efsInternal is not able to "look" for a global vars declared/calculated in efs.

Can anybody explain if there is such restriction, or I am doing something wrong here?
Note: I did try to declare a local var inside MyOscF, but still does not work.

Thank you.

[eSignal_JasonK]

Hello Mihai,

There aren't any restrictions. What you have to keep in mind is that efsInternal() receives a unique instance of any global variables. Therefore, your first function sees AvB as undefined. You can either assign a value to AvB within the called function or pass the value to the function as you did in the second efsInternal() example. Thanks for bringing this to my attention. I've added a note about this to the KB.

Try the following code. You will see that either method can be used.

var AvB;
var stMA1;
var stMA2;
var stOsc;
var bInit = false;

function main() {
    if (bInit == false) {
        AvB = 2;
        stMA1 = sma(10);
        stMA2 = sma(20);
        stOsc12 = efsInternal("MyOscF",stMA1,stMA2); 
        //stOsc12 = efsInternal("MyOscF1",stMA1,stMA2,AvB);
        bInit = true;
    }

    var nstOsc12 = stOsc12.getValue(0);
    
    return nstOsc12;
}

function MyOscF(stMAf,stMAs) {
    if (AvB == null) AvB = 2;
    return (stMAf.getValue(0) - stMAs.getValue(0)) / AvB;
}

function MyOscF1(stMAf,stMAs,XX) {
    return (stMAf.getValue(0) - stMAs.getValue(0)) / XX;
} 

[mbuta]

Hi Jason,

Again, a very competent and to the subject response. Thank you!

Now, you can see the real issues with this [that maybe development team wants to look at for future releases]:

We must pass [not use in the body of the function] every constant and/or var, which kindof defeats the purpose of efsInternal, because it is virtually impossible and inpractical for real life systems.
You want to be able to use the results of other calculations, even if they are not necessarily data series.

In my case, I can't refer to a constant [because my AvB is a constant calculated at ALLBARS time and never changes].

This, along with the other suggestion I made many times [to allow us to control how often a study is executed [now they execute on every tick]], would make efsInternal very useful and PRACTICAL.
Note: Every attempt I made to use it failed, due to perfomance degradation [at load time, which for me is essential].

Again, thank you for your response.

[eSignal_JasonK]

Hello Mihai,

Originally posted by mbuta
Hi Jason,

Again, a very competent and to the subject response. Thank you!

You're most welcome.

Now, you can see the real issues with this [that maybe development team wants to look at for future releases]:

We must pass [not use in the body of the function] every constant and/or var, which kindof defeats the purpose of efsInternal, because it is virtually impossible and inpractical for real life systems.
You want to be able to use the results of other calculations, even if they are not necessarily data series.

In my case, I can't refer to a constant [because my AvB is a constant calculated at ALLBARS time and never changes].

As I showed in the previous code example I gave you, you do not have to pass every constant to the called function. You can establish the value for the constant at ALLBARS directly in the body of the function that is called by efsInternal(). Take a look again at the MyOscF() function.

function MyOscF(stMAf,stMAs) {
    if (AvB == null) AvB = 2;
    return (stMAf.getValue(0) - stMAs.getValue(0)) / AvB;
} 

The first line of that function that checks to see if AvB is null and then assigns a constant value to it is the same process as checking for ALLBARS. In the code example I assigned a hard-coded constant, but it could also be another function.

function MyOscF(stMAf,stMAs) {
    if (AvB == null) AvB = calcMyValue(); // this only happens once and is just a value, not a Series.
    return (stMAf.getValue(0) - stMAs.getValue(0)) / AvB;
}

function calcMyValue() {
    return 2 *1;  // or whatever your constant needs to be
} 

This, along with the other suggestion I made many times [to allow us to control how often a study is executed [now they execute on every tick]], would make efsInternal very useful and PRACTICAL.
Note: Every attempt I made to use it failed, due to perfomance degradation [at load time, which for me is essential].

Again, thank you for your response.

Are you implying that there is some inherent malfunction with efsInternal()? This function has been heavily tested and we believe it to be as efficient as it can be. If you post a complete working example that demonstrates the performance degradation that you're experiencing I'd be more than happy to investigate.

A couple things you may want to consider that could help improve performance at load time is to limit the amount of data that needs to be loaded by the chart with a custom time template. If your study is also accessing external time frames, limit the amount of data for those time frames as well in the same time template. Also, within the body of your functions called by efsInternal() you can minimize processing by encapsulating routines inside a check for BARSTATE_NEWBAR to update the series return value(s). Store the returned value(s) in global variables and just return the previously calculated value when the bar state is anything other than new bar. The Series will still be calculating on every tick in real time, but will just return the last calculated result, which is much more efficient than reprocessing all the code in the called function if it is not necessary to do so. Hope this helps.

[mbuta]

Hi Jason,

I understand what you say.
For constants is ok. For vars is not.

Thank you.

[eSignal_JasonK]

Hello Mihai,

Originally posted by mbuta
Hi Jason,

I understand what you say.
For constants is ok. For vars is not.

Thank you.

Again, you're most welcome.

For vars, you should also be able to accomplish the same task as with constants. Depending on what the var is storing, you may just need it to be in the form of a series. If that's the case run it through an efsInternal() call within your MyOscF() function as well. In the previous code example the calcMyValue() function would be called by efsInternal() to convert it to a series. The calculation would need to be something other than a constant of course. If you give me an example of the type of var or calculation you are referring to I'd be more than happy to give you a code example to demonstrate the process.

Alternatively, the var you're referring to that may be a series could be established in main() and then passed to MyOscF() as a parameter, which ever you prefer.

[mbuta]

Hi Jason,

Yes, I know you can pass a var, if it is a series but, like I menstioned before, this means execution on every tick, which affects the performance [if many enough and multiple charts with 6-7 instances of the efs].

Which brings me to an other question I have [if you prefer, I can start a new thread].

Is there anything else than reloadEFS() that can generate additional calls to main?

I am using exactly the same efs script as PriceStudy and NonPriceStudy and the NonPriceStudy mains are called at least twice as much as those in Price pane.
Note: I know that EFS2 in external timeframes may do that, but we talk aaples and aaples [same script, same time intervals], but one gets called twice as much.

I checked and I do not have any active reloadEFS [other than those executed to change the configurations, when I press buttons and which execute only at ALLBARS time].

Any idea why?

I experience significant performance degradation on active stocks [QQQQ, INTC, CSCO, AAPL, etc.] although my average execution time is .3-.4ms. Sometimes [on volume spikes] takes up 1-2 min to "recover" and sometimes looks like is looping forever.

Thank you.

[eSignal_JasonK]

Hello Mihai,

Originally posted by mbuta
Hi Jason,

Yes, I know you can pass a var, if it is a series but, like I menstioned before, this means execution on every tick, which affects the performance [if many enough and multiple charts with 6-7 instances of the efs].

Within the body of your functions called by efsInternal() you can minimize processing by encapsulating routines inside a check for BARSTATE_NEWBAR to update the series calculation once per bar. Store the returned value(s) in global variables and just return the previously calculated value when the bar state is anything other than new bar. The Series will still be executing on every tick in real time, but will just return the last calculated result, which is much more efficient than reprocessing the entire calculation on every tick. Have you tried to implement this yet?

Which brings me to an other question I have [if you prefer, I can start a new thread].

Is there anything else than reloadEFS() that can generate additional calls to main?

Only a user-defined function that contains direct calls main(). However, if you're using the exact same code for both price and non-price pane, the number of executions should be the same.

I am using exactly the same efs script as PriceStudy and NonPriceStudy and the NonPriceStudy mains are called at least twice as much as those in Price pane.
Note: I know that EFS2 in external timeframes may do that, but we talk aaples and aaples [same script, same time intervals], but one gets called twice as much.

I checked and I do not have any active reloadEFS [other than those executed to change the configurations, when I press buttons and which execute only at ALLBARS time].

Any idea why?

I've never seen this behavior before. If you post the formula you're having this problem with, I'd be happy to take a look.

As a quick test, I used the following code on INTC. The code counts executions and returns the number to the cursor window as a string.

function preMain() {
    setPriceStudy(true);
    setStudyTitle("test");
    setCursorLabelName("Executions", 0);
    //setPlotType(PLOTTYPE_HISTOGRAM, 1);
}

var bInit = false;
var nCount = 0;

function main() {
    if (bInit == false) {
        nCount = 0;
        bInit = true;
    }
    
    if (getBarState() == BARSTATE_NEWBAR) nCount = 0;
    nCount++;
    debugPrintln("Count: " + nCount);
    
    return nCount+"";
} 

Here's what I see on my end (animated gif).

I experience significant performance degradation on active stocks [QQQQ, INTC, CSCO, AAPL, etc.] although my average execution time is .3-.4ms. Sometimes [on volume spikes] takes up 1-2 min to "recover" and sometimes looks like is looping forever.

Thank you.

This may not necessarily be related to EFS. If you post a formula that I can test on my end, I'd be more than happy to do so.

[mbuta]

Hi Jason,

Very neat test you have there on INTC, that runs in real time from this board.

1. As I aleady said, I just wanted to make sure that there is no other [hidden] possibilty for reloading efs.
Note: I have a mixture of EFS2 and EFS1 studies, and I was reloading efs at higher intervals NewBar [just to redraw the charts].
I eliminated them, now number of executions is the same [at least last few days], but I still have the performance problem with busy stocks and peak volumes.

2. Re. Using NewBar test in my functions: Yes, I am aware of that [Steve showed my the trick some time ago], and I do it with the functions that are ok to stay the same until new bar [states, etc.].

I can't do it for all of them [all my raw are indicatorts "normalized"] because I want to keep the flexibility to run them more than once per bar.
For that I calculate a "TimeToUpdate" when I want to update the indicators. In between I return the last values, which I store directly in the ReturnArray.

I would die to be able to use efsInternal, but all those .02ms [the time of ANY basic call] kill me.
When [if ever] I will finish my work, the plan is to run only one instance of the code and display the results only. Then, it should be no problem, but I am still far from that.

3. Re. Posting my code. As you know, it is not the fact that I do not want to post it, but it is [really] such a mess that I would be completelly inconsiderate to ask someone to even try to understand it [even a qualified consultant].
That is why I try to ask specific questions, than go and bang my head myself against my own mess.
You have been very understanding and helpfull.

Thank you.