getHours/getMinutes major performance drag?

[mbuta]

Hi everybody,

As you may know, I have been crying about performance for a while.

I just found one line of code that [if eliminated] reduces my execution time in half [by more than .2ms] and I want to share it with others that may have similar problem.

var CurrentTime = getHours()*100+getMinutes;

Just one line, hard to believe this function [which I had forever] has such a great impact on execution time. But if it is true, maybe it is worthwhile publishing a list with overheads introduced by various function. It would save a lot of aggravation to many of us.

In the same idea, please be aware of the following performance impact: for whatever reason, for some functions, if you not specify BarIndex [like close() instead of close(0)] it may impact the execution time.
Moreover, some functions may even give strange behaviour of your efs. Don't remember which ones [I had a thread about this].

Mihai Buta

[eSignal_JasonK]

Re: getHours/getMinutes major performance drag?

Hello Mihai,

When you are experiencing a problem with your formula code, you need to post a working example that clearly demonstrates the problem. We're more than happy to help test your code and provide you with a solution, but it's virtually impossible to do this without seeing exactly how your code is written. The information you've posted is misleading to others. The problems you are experiencing are most likely related to your specific code.

Originally posted by mbuta
Hi everybody,

As you may know, I have been crying about performance for a while.

I just found one line of code that [if eliminated] reduces my execution time in half [by more than .2ms] and I want to share it with others that may have similar problem.

var CurrentTime = getHours()*100+getMinutes;

This by itself, is not a problem. Calling it on every tick in real time would require a fractional amount of extra processing. To eliminate that, make CurrentTime a global variable and only set its value inside a check for BARSTATE_NEWBAR. That will only allow it to execute once per bar rather than every tick.

Just one line, hard to believe this function [which I had forever] has such a great impact on execution time. But if it is true, maybe it is worthwhile publishing a list with overheads introduced by various function. It would save a lot of aggravation to many of us.

This in an unrealistic request because 99% of the time, formula problems are related to specific combinations of code written by the user rather than a specific function on it's own. The possible combinations are virtually unlimited.

In the same idea, please be aware of the following performance impact: for whatever reason, for some functions, if you not specify BarIndex [like close() instead of close(0)] it may impact the execution time.
Moreover, some functions may even give strange behaviour of your efs. Don't remember which ones [I had a thread about this].

Mihai Buta

There is not a problem with close() vs. close(0). The behavior you're looking at should be expected. You just need to understand what is happening. When you use close(), you are creating a series object. When you use close(0), you are only asking for the current value of the close series, which requires less processing than creating the series object. You can use close() directly in an expression and the EFS2 engine knows to use the current value of the series to perform some math. This was done so that EFS1 formulas would be backward compatible in EFS2. Using close(0) would be more efficient and is the proper way to use the function when creating an expression. Furthermore, calling close(0) multiple times throughout your formula is less efficient than assigning close(0) to a local variable and using the variable throughout your code. You should only use close() when you need to pass this as a source to another study.

There are many things that can be done to help improve the performance of your code, but without seeing some working examples we can not give you the specific solutions you need. Please post some formula examples and I'd be more than happy to take a look.

[ACM]

Mihai
To add to Jason's reply I just completed running a series of measurements on the execution times of
(getHour()*100)+getMinute();
and the highest reading it registered in the Performance Monitor under any condition was 0.07ms so I find it difficult to understand how removing that one line alone would save 0.2ms. You may want to post a sample script that demonstrates this one issue.
Regardless - and just to maintain a sense of perspective - it should be noted that we are talking about 2 tenths of a thousandth of a second
Alex

[mbuta]

Guys,

I was sure I will be made look like an idiot, again.
It happens EVERY time when potential shortcomings of eSignal are reported.

For the record: I eliminated that one line of code and I cut my execution time [for the rest of 4000] in half. From about .4ms to less than .2ms.

And this is not misleading anybody, this is a fact.


Nobody makes perfect products, nobody, not IBM, not Cisco, not Oracle, not Gm, not eSignal. The diference is how you deal with shortcomings, some recall 100k vehicles and fix it, some deny there is any problem.

Sincerely,
Mihai Buta

use of getHour() and getMinute() is efficient

Hi Mihai,

I am confused about what you are saying. Specifically, the line of code you eliminated (getHours()*100+getMinutes;) contains two commands that are invalid. Specifically, getHours()should be getHour() and getMinutes should be getMinute().

Based on your post, I created a small efs in an attempt to duplicate what you are saying. I could not duplicate it. However, I have attached a copy of what the efs eventually morphed into. This efs times the execution of this line a = getHour(0)*100+getMinute(0); Personally, I always specify the bar index as I have found it to be faster because it returns a numerical value versus an object.

Please do not interpret this statement as indication that use of getHour() and getMinute() are not efficient, as they are very efficient as well. btw, the nature of these different variables (number versus object) may be determined easily enough by using the JavaScript command typeof(). As an aside, I do not believe this command likes series objects. Each time I try querying their type, my computer lets its feelings towards me known.

When I ran this on my machines, the average execution time (over 100,000 executions) was 0.00922 milliseconds. This number is similar to what Alex mentioned and is consistent with my experiences with both small and much larger, more complex, efs's.

Here is an excerpt of the output from another efs where I was testing getHours()*100+getMinutes; (error in line 26). Additionally, it contains the output of the enclosed efs:

gdFunctions.efsLib, line 26: ReferenceError: getHours is not defined

17: time = 1129250128031
25: time = 922 ==> 0.00922 milliseconds per call using getHour(0)*100+getMinute(0) 
           which is 0.00000922 seconds per call 

As to the use of close() versus close(0), you can easily adapt my efs to confirm what Jason indicated about these functions as well.

function preMain(){
 
 setPriceStudy(true);
 setShowTitleParameters(true);
 setStudyTitle("test");
 setShowCursorLabel(false);

}


var binit = false;
function main(){
 if (binit == false ){
  binit = true;
  var b = new Date().getTime();
  debugPrintln("17: time = "+new Date().getTime());
  
  var a;
  for (var i=0;i<100000;i++){
   a = getHour(0)*100+getMinute(0);
  }
  
  b = new Date().getTime()-b;
  debugPrintln("25: time = "+b+ " ==> "+b/100000+
" milliseconds per call using getHour(0)*100+getMinute(0) which is "+b/100000000+" seconds per call");
 }
} 

[eSignal_JasonK]

Hello Mihai,

We would sincerely like to help you solve your problem. Unless you can post a formula example that clearly illustrates the problem, we can not help you or confirm any shortcomings.

[mbuta]

Why don't you guys try this efs and see for yourselves the diference.
Run it with and without line 58.

Notes:
1. It is executed only on NEWBAR.
2. They are getHour and getMinute. And I have no clue about objects vs numbers. I am a novice user, sorry.
3. I was just reporting the issue, when I solved the problem [one of which I cried for help for months].
4. Jason, I still have strange reloads. I am using "isInSession" and I suspect it is from there, but not sure yet.


http://share.esignal.com/ContentRoot/Mihai%20Buta%20-%20Multi%20Time%20Interval%20Indicators/AllOBOS.TestGetTime.efs

[ACM]

Mihai
I ran two versions of the efs in Tick Replay over 5 days of 1 minute data and the results of the tests are shown in the following images. AllOBOS[1] is the version posted in the File Share group while AllOBOS[2] is the same but with line 61 (not 58 btw) commented out.





As you can see over the course of 5 days there is essentially no difference between the execution times of the two versions. FWIW these tests were repeated multiple times and returned consistently similar results. Also, in order to make sure that in the test one efs would not interfere with the other each script was run on its own.
I also ran some tests to see the execution times on first load and the results are shown in the following images.





Here the average difference between the two is around 0.07ms which is the same value I mentioned in my prior reply and certainly nowhere near the 0.2ms you state in your first message (see first quote below).
Even in this case I ran multiple tests and although the difference was on average 0.05ms (and sometimes even less) I am posting the worst result out of fairness.
Also, if instead of getHour()*100+getMinute() one used getHour(0)*100+getMinute(0) (as shown in the examples in this article of the EFS KnowledgeBase) the difference would be even smaller (less than 0.01ms).
At this point I can see no differences that are of any relevance neither on first load nor when running the scripts over extended periods.
Alex

PS. FYI the link you posted for the efs is not working

Originally posted by mbuta
I just found one line of code that [if eliminated] reduces my execution time in half [by more than .2ms] and I want to share it with others that may have similar problem.

var CurrentTime = getHours()*100+getMinutes;

Originally posted by mbuta
Why don't you guys try this efs and see for yourselves the diference.
Run it with and without line 58.

Notes:
1. It is executed only on NEWBAR.
2. They are getHour and getMinute. And I have no clue about objects vs numbers. I am a novice user, sorry.
3. I was just reporting the issue, when I solved the problem [one of which I cried for help for months].
4. Jason, I still have strange reloads. I am using "isInSession" and I suspect it is from there, but not sure yet.


http://share.esignal.com/ContentRoot/Mihai%20Buta%20-%20Multi%20Time%20Interval%20Indicators/AllOBOS.TestGetTime.efs

Hi Mihai,

I had some difficulty finding your code as well, however, once located, I found some issues that you may want to address. I figure they are just oversights on your part, but based on your own comment regarding your level of expertise, they may be of some assistance. (btw, since JavaScript is an object oriented language, it may help to learn the differences between objects and numbers, it certainly helped me understand the original EFS structure. It is not required so much in EFS2, however, I still think it may be helpful.)

First, although it is often unnecessary, it is good programming style to declare variables before using them, either as global variables outside main or as regular variables within main. For example, neither nCurrentIDX nor CurrentTime were defined using a var declaration. There were several others as well. This can adversely affect you, especially in larger efs's, where you are allowing JavaScript to decide for you if they are new local variables or previously defined global variables. Second, there was an undefined function in there (PrevDayPivots) and the cci formula had a variable in there that while defined (SrcPrM0), was not assigned any value.

I had not tried to run and time the efs, rather, I just looked at the structure. Since Alex documented his test method pretty well, I think I can run similar tests to validate them for you Mihai.

Mihai,

I went ahead and ran tests similar to what Alex had done, trying to format them similarly as well. It looks as if I ran them in the reverse order with the times for the initial load being the first two and the Tick Playback being second set.

The timings were run with the [1] file unchanged (first load 0.09887) from download and the [2] file (first load 0.02133) is with the CurrentTime line commented out. As you can see, the duration in miliseconds, when loading approximately 3000 bars, was very similar to what Alex had come up with.

With Tick Replay, I used last weeks ES #F data, where the [1] file is unchanged and the [2] file is with the CurrentTime line commented out. The results there were 0.6032 and 0.5884 miliseconds per call respectfully. These numbers are similar as well.



I hope this helps you figure your problem out.

[mbuta]

Guys, thank you.

Sorry you had har time finding the efs. It is the only way I know to upload, through my File Share.

Steve, I do have all var declared. What I posted was a short from several thousand line. Probably, overlooked [but I did run before posting].

Alex, you were right about (0). Tha was the problem. With (0) I get similiar results with yours [actually, better -> .03-.05].

Thank you.