EFS2 Custom Data Series - Perfomance Issues

[mbuta]

Hi,

I need a few clarifications about custom data series feature of EFS2. If anyone can help, it is appreciated.

1. Is it true that data series of EFS2 built-in studies/series don't need to be declared [like we had to do with studies in EFS1], and still can access individual values?

2. Is it true that custom data series [declared/created with efsInternal] are created on first bar [like regular studies] and then can be accessed/referenced at any time, for any bar?

3. Is it true that we can create custom data series using, as source, an other custom data series [nested custom data series]?
3a. Are there any restrictions/rules [other than passing the data series source as the last parameter] if this is possible?

Thank you.
Mihai

[ACM]

Mihai
With regards to point 3 the answer is yes and as an example you can see the script in this post (and the one on the same subject later in the same thread)
Alex

[mbuta]

Thank you Alex.

Anybody on the first two questions?
The examples in documentation are [for me] confusing, if one wants a data series for more than plot only.

Thank you.
Mihai

[ACM]

Mihai
As to your point 1 you can retrieve a single value directly (ie without declaring the study) by defining the bar index as the last parameter. For example if you want to retrieve the value of two bars ago of a 10 period simple moving average you would write
var x = sma(10,-2)
With regards to point 2 if I understood your question correctly then yes they behave like regular builtin studies.
Alex

[mbuta]

Hi Alex and thank you for your replies.

I will explain in more details what I want to accomplish, maybe you can tell me to what extent it can be done.

I have been using multiple time intervals for long time [see some examples on my File Share folder], but I also "normalize" the indicators, which allows me to combine odd indicators.
For example I can do
BaseMx=(CCI+MARSI+Osc+MACD)/4
BaseOB=(CCI+Stoch+RSI+(PDI-NDI))/4

These normalized indicators and their derivatives I would like to decalre them as data series, so I can easily retrieve them [to detect divergencies, basedon Pivots, for example] and this is not easy to do in EFS1.

An other reason I would like to convert to EFS2 is because it fills back to the first bar, which cannot be done in EFS1.

In the same time though, I am very concerned with performance, because I am running many windows with 4-5 panes [for now] in each window.

I did try to convert my M1/M2 timeframe indicators to EFS2 and the execution time increased dramatically. The reasons are simple:
a/ EFS2 studies cannot be declared in pre-main, where they are executed in C, rather than Java and
b/ In EFS2, each component of the study needs to be declared independently [Signal, Histogram, Macd; or ndi, pdi, adx; etc.]

I am sure that these issues are known and common and the question is: What is the best way to convert to EFS2, with minimum [or no] performance penalty?

Thank you for your time.
Mihai

[eSignal_JasonK]

Hello Mihai,

For your BaseMx and BaseMo series, you would use efsInternal(). This function calls another user-defined function within the EFS and creates a data series object from its calculation.

function main() {
    var BaseMx = efsInternal("calcBaseMx");
    return BaseMx;
}

function calcBaseMx() {
    return (cci(10)+osc(10, 21, false)) / 2;
} 

BaseMx is now a series object, so you can reference its previous values with the .getValue(index) method.

var prev = BaseMx.getValue(-1);


As for performance, if you are creating the exact same processes with EFS2 code to replace EFS1 code you shouldn't notice any differences in performance. It may be more efficient in some cases, depending on how you wrote your EFS1 formulas. When introducing multiple time frames into your new formulas, you are also adding processing instructions, which will most likely add some level processing requirements. You should expect to see this due to the increased processing demands that are being made by your code. If you use custom time templates to control the amount of data loaded into your charts you can help minimize the amount of processing requirements during the initialization of the formula.

Whats the best way to convert to EFS2? It really depends on the specific formula in question and what its needs are. Post some EFS1 code examples along with your EFS2 conversion of that code and I'll see if there are any suggestions or improvements we can make.

[mbuta]

Jason thank you for your reply.

I understand what you suggest because it is exactly what I tried and ran into problems, here is why:

a/ I have already the components declared as studies, as I use them in many other place than BaseMx. I want to use them rather than declare them again.

b/ I have the same thing for three diferent time intervals and I don't know how to do the efsInternal for multiple time intervals.

c/ If I declare them in the body of main(), I get dramatic performance impact. I tried to do it only at ALLBARs and I ran into problems again [strange manifestations, I need to investigate why].

d/ Like I already said, my raw indicators [CCI, Osc, etc.] are normalized to diferent values [either a TrendLevel, like CCI, ADX, or an OB/OS normalization, like Stoch and RSI, or to other reference for Osc, MACD and MA-RSI].

Right now, in EFS1, I declare all studies at ALLBARS time and calculate on each bar thereafter. I love the idea of data series, and I converted most raw studies, must I was not able to use efsInternal yet.

To summarize, maybe you can point me to some examples that (a) use efsInternal for calculated values, without having to declare the studies inside my function and (b) it is done such that the data series is declared once and used on each bar.

One more thing. Is there a way to declare macd with simple MA's in EFS2, because the builtin uses ema, if I am not mistakes.

Thank you Jason,
Mihai

[eSignal_JasonK]

Hello Mihai,

a) Depending on the specifics of your code, you may or may not be able to reuse the studies that you have already declared. If you are created multiple data series objects that are controlled by different intervals you will need separate instances of the built-in studies for each. If you post some completed code examples of what you're trying to accomplish I can give you some more specific guidance.

b) The easiest thing to do is to pass your desired interval to the study functions inside the functioned called by efsInternal. Here's what it would look like using the previous code example I gave you. Notice the inv() function inside the parameters for the cci and osc.

function main() {
    var BaseMx = efsInternal("calcBaseMx");
    return BaseMx;
}

function calcBaseMx() {
    return (cci(10, inv(15))+osc(10, 21, false, inv(15))) / 2;
} 

Now BaseMx is based on a 15-minute interval. If you need another BaseMx series that's based on yet another interval, create a BaseMx2 and set it to it's own efsInternal call.

function main() {
    var BaseMx = efsInternal("calcBaseMx");
    var BaseMx2 = efsInternal("calcBaseMx2");
    return new Array(BaseMx, BaseMx2);
}

function calcBaseMx() {
    return (cci(10, inv(15))+osc(10, 21, false, inv(15))) / 2;
}

function calcBaseMx2() {
    return (cci(10, inv(30))+osc(10, 21, false, inv(30))) / 2;
} 

c) This is difficult to answer in general terms. It really depends on the specific code you are using. You shouldn't notice a dramatic difference in performance. There may be something you're not doing quite right. Post some code and I'll take a look at it.

d) With efsInternal, the studies that are not being declared inside the called function are not being reinitialized on every execution of the formula. I think that is what you think is happening based on how EFS1 code logic works. The way the EFS2 engine uses the efsInternal() function is a new behavior. Essentially, those studies declared inside the efsInternal function are only initialized once at the first iteration. After that the EFS2 engine has all the info it needs to produce the calculated result for every bar in your chart. The result is a series object that is in the same structure as one of the built-in studies, which also create series objects. If each efsInternal function is based on a different interval you will have to use separate study functions declared inside each efsInternal call. I hope this make sense. Again, if you post some code examples that illustrate the process you are working with I can give you some more specific guidance.

As for the simple vs. exponential option on the macd studies, I've sent an inquiry to development about that. You should also send in a suggestion for adding that as an option to [email protected].

[mbuta]

Jason,

Thank you. As always, your comments are clear and to the point.

Can you please elaborate a bit on points (a) and (d)? Or point me to some document that describes who EFS2 engine work in more detail?

As for posting "completed code", sorry, but I have one big mess efs file, wgich includes all the idicators I use and display them with buttons [see some examples below]. But, essentially, I have a bunch of studies declared at ALLBARS time, then I update them on every NEWBAR. Some of them are processed [normalized]. I display them as I want, in PriceStudy or not.

Two more question for you:
1. Because I update at NEWBAR, my plots are one bar late. Is there a way to do a setBar for an entire array of plots, or need to do each one of them individually [case in which, i give up].
Note: I made it work in EFS1, but not in EFS2.

2. In EFS2, ref(-1) on a diferent time interval data series, refers to chart previous bar, or of that time interval?

Thank you again.
Mihai

[ACM]

Mihai

Originally posted by mbuta
As for posting "completed code", sorry, but I have one big mess efs file, wgich includes all the idicators I use and display them with buttons [see some examples in MultiTimeInterval file sharing folder - don't know how to upload here].

When you compose a message/reply scroll down on that page and you will find an Attach File section. Click on the Browse button, navigate to the folder on your computer that contains the efs and select it. You can attach one file per message.
You can also post your code just by copying and pasting it into the body of your message. In this case it would be preferable to enclose it in [ php ] your code [ /php ] or [ code ] your code [ /code ] tags (without the spaces in the tags) so that it is properly aligned.
Alex

[mbuta]

Thank you Alex,

Looks like I manged to upload some images. About uploading code, I can't expect anybody to understand my mess.

Mihai

[eSignal_JasonK]

Hello Mihai,

a and d) I think I've pretty well covered how the EFS2 engine uses a series object. There isn't anything else I can describe for you that will help you solve your problems. The only way at this point to help get a coding solution for you is to work with some specific code (see point 6 in the EFS Studies posting guidlines). If your study has a lot of code, the best thing you can do is try to create a simplified version that just isolates the problems your are trying to fix. Going through this process will often point out the specific problem and help you resolve your own issues. It's going to take some effort on your part, but its really the only way we can help you. We have to see and test the specific coding processes you are using in order to help provide a specific solution.

1) Yes you can, setBar( type, barIndex, arrayOfValues ).

setBar(Bar.Value, -1, aYourArray);

I would avoid setBar altogether, however. Just let your indicators plot on every tick and then surround only the code you want to restrict execution on inside a BARSTATE_NEWBAR block. That way the last calculated value of the indicators are plotted before newbar occurs.

2) The chart's previous bar.

[z11]

Hi Mihai,
Looks like you are doing something very interesting.
I am trying to draw some buttons like you are in the price pane... but can't figure out the vertical position ...would you share with me the x y values of say, the first two buttons, Ch64 and MA50? Do you use drawTextRelative or drawTextAbsolute?

Thanks,

ziggy

[mbuta]

Hi Ziggy,

I am using DrawTextAbsolute. Look at my MultiTimeInterval File Sharing folder. I posted the entire Buttons Engine, the Plot Engine [Price and NonPrice Study] and all the support functions needed to color the studies.
It allows you to use the same script for [almost] anything you want to do. You can plot up to six diferent sets of indicators per each button. M0/M1/M2 are the diferent time-intervals.
Please study the EvenHandlers to understand what/How can be plotted, then replace my buttons and vars with yours.

Note: EFS2 lets you do the positioning much easier [a number of rows from the margins of the chart]. Look at the description of the new drawing functions.

Good Luck!
Mihai

[mbuta]

Hi Jason,

I know you are right [about trying to some examples, or isolate my problem] and that is why I am trying to better understand how the EFS2 engine works, particularly, on multiple time intervals.

For example, I wanted to understand if a studyX(seriesY, inv(15)) an a 5 minutes chart is updated every 15 minutes, or every 5 minutes. From your response, I gather that every 5 minutes [every bar].

I also wanted to know if I can declare a new custom series, using the already calculated values of an other series.
For example: rsi(14, getValue.studyMA20M2, inv(M2))
where studyMA20M2 is the MA20 at M2 time interval.
From your response I gather that this is not possible and I have to redeclare the MA20M2 inside rsi, like
rsi(14, sma(inv(M2).

Note: In EFS1 I do as follows: rsi(14*M2, getValue(xx), but this does not fill the begining of the chart and forces me to use min 600 bars charts [which I hate], for last 2-300 useful bars. Studies like CCI need a lot of bars.

Is my understanding correct?

About setBar. I aggree that better avoid it and I do let my indicators plot every tick, but I update them on NEWBAR only [on every tick is not practical for busy markets] and at that time I get the correct close values of the previous bar. Therefor, I must redraw last bar.
Like I already said, I made it work in EFS1, but not for indicators declared in EFS2 (?!?). Values can be updated as an array, but colors I have to do it individually [TBD later].

Thank you again,
Mihai