EFS useage monitor

[fish7]

I have a workspace that has several charts and EFS scripts running and is chugging a lot of CPU. Is there any kind of debug monitor that will provide the amount of CPU time each EFS script was using over a given time period? Such a tool would really be helpful for me in figuring out where to focus my efforts on optimizing my code.

Hi f1sh7,

Yes there is. Under the tools...EFS drop down windows, select the Performance Monitor. During testing, I recommend you verify your results using tick playback. You may also want to double-check the results while running a single efs in tick playback with the Performance Monitor.

[fish7]

That's super!

Thanks!

Hi f1sh7,

Your most welcome.

Be careful, use the results as a guide, as they are not exact. However, the longer the performance is measured, the more representative the results.

I'll usually run individual efs's in tick playback at 100x speed (making sure cpu load is <50%) through 1 or 2 days of market action.

[fish7]

Steve,

Currently, the choices for EFS scripts computation are ComputeOnClose or compute every tick. .. correct?

Is there a function (or a clever set of lines to add to the beginning of an EFS) to put into the pre-main that would allow users to specify that an EFS is run only every "n" ticks?...

Some of my EFS scripts must be calculated intrabar, but I would be happy if they ran, say, every other 5 or 10 ticks. that would cut their CPU useage dramatically, yet give me ample information.

JOHN

John,

You are correct with regards to the available script execution options.

As far as a clever set of steps, that can be done. However, you would have to be careful to not defeat the tick and new bar logic associated with the efs. Keep in mind that you still have to return data to the chart every tick as well.

In terms of difficulty, ensuring the logic is not affected can be tough. Returning data to the chart every tick is relatively straightforward.

As you mentioned earlier, looking for efs's that are resource hogs is a good first step.

Originally posted by f1sh7
Steve,

Currently, the choices for EFS scripts computation are ComputeOnClose or compute every tick. .. correct?

Is there a function (or a clever set of lines to add to the beginning of an EFS) to put into the pre-main that would allow users to specify that an EFS is run only every "n" ticks?...

Some of my EFS scripts must be calculated intrabar, but I would be happy if they ran, say, every other 5 or 10 ticks. that would cut their CPU useage dramatically, yet give me ample information.

JOHN

[fish7]

Steve,

I coded up an EFS that colorizes the volume bars to red (from black) when the volume exceeds a user-selectable threshold level. I found that this uses the CPU a lot, so I put a counter loop in the script so that the volume bar is only update every "n" ticks (default is n=5). This is built for the YM M8 (dow e-Mini) on 5min charts.

What I find odd, is this: While the volume bar is forming, and is over the threshold where I want it colored red, the bar color toggles back and forth between red and black on every tick. Try it in bar replay and you will see.

I do not understand this. It is as if eSignal re-sets the bar foreground color to the default after every run through an EFS script, and then re-sets it to what it should be as the script runs. If I am right in this, this is one reason why this script is so computationally intensive -- all that unnecessary flipping of bar foreground color back and forth.

Is there something I am doing wrong here? You understand (I hope) what I am trying to do. I just figured the bars would go to red once the volume for a given bar exceeded the threshold level.

Thanks, in advance, for any help.

Hi John,

I marked up your script as best I could. I played back at 200x speed, seemed to work well.

my general comments or comments on changes I made will be marked with a squiggly line //~

hope this helps.

/********************************************************************
Tis EFS colors the volume bars red when they are above a
threshold level set by the user.
To lessen the CPU useage, this EFS employs a counter, and 
the bar is only updated every "n" ticks (where "n" is user-selectable
*********************************************************************/
 
var fpArray = new Array();

//~ this variables is a parameter supplied to main by the efs Engine, provided by your FunctionParameter objects below
//~ it should not be declared as global variables
//~ var LargeVolThreshold;


//~ var NumOfTicks = 0;   //~ commented out, I used 'counter' global below

function preMain() {

 setPriceStudy(false);
 setStudyTitle("Volume Threshold Colorized");
 setCursorLabelName("Vol",0);
 setDefaultBarFgColor(Color.black,0);
 setPlotType(PLOTTYPE_HISTOGRAM,0); 
 setDefaultBarThickness(2,0);
 setShowTitleParameters(true);
 askForInput();
 var x=0;         

 fpArray[x] = new FunctionParameter("LargeVolThreshold", FunctionParameter.NUMBER);
 with(fpArray[x++]){      // Vol above this is large, but not Mega
   setLowerLimit(2);  
   setDefault(2000);
 }
 fpArray[x] = new FunctionParameter("CalcEveryNticks", FunctionParameter.NUMBER);
 with(fpArray[x++]){      // Calculate every n ticks
   setLowerLimit(1);  
   setDefault(2);
 }    
 
 setShowTitleParameters(true);    
 //~ NumOfTicks = 0; //~ commented out, I used 'counter' global below

}

var last=null; //~ new global variable that retains the last color

var counter=0; //~ new global variable that retains thecounter value
function main(LargeVolThreshold, CalcEveryNticks) {

 // Only do this if it's a new bar
 if (getBarState() == BARSTATE_NEWBAR) {
  last=Color.black;
  counter=0;
  setBarFgColor(last); 
 } 
 
 //~ ALWAYS use a bar index within the parenthesis in these methods (high(0),close(0), volume(0), etc,) else you will waste resources
 //~ call this method only once per tick, each additional time you call it you are wasting resources
 //~ local variables are very very efficient and should be used instead of calling any method twice
 var myVolume=volume(0);

 // This thing chugs the CPU... so only update drawing the colorized bar, every so many ticks.
 // decrement the counter every tick.
 if(counter--) {  //~ the counter value is tested first, then decrimented.  A non zero is true, a zero evaluates to false
 
 setBarFgColor(last); //~ this must be called every tick, otherwise 'setDefaultBarFgColor(Color.black,0)' in preMain determines color
  
 // Just abort, and report the volume to the cursor window
 return myVolume;  //~ local variables are very very efficient and should be used instead of calling any method twice

 }
 else{//~ Re-set the counter to CalcEveryNticks
  counter=CalcEveryNticks; 
 }


 if(myVolume>=LargeVolThreshold){ //~ local variables are very very efficient and should be used instead of calling any method twice
  last=Color.red; //~ new global variable that retains the last color
  setBarFgColor(last);       //Vol is over the threshold, so now color the bar red
 }
 
 return myVolume; //~ local variables are very very efficient and should be used instead of calling any method twice
} 

[fish7]

Steve,

Wow... thanks!... that was full of useful ideas.

I did a little test, and compared your faster version w/my original script, and it ran using 1/3rd the CPU time. That's amazing!

I had no idea that there would be a resource (and time?) difference between using close() versus close(0), especially since they return the same thing.

Now I will go and re-check all my EFS's and use more local variables. Did not realize that calling those functions more than once is slower than declaring a local variable.

Thanks a TON !!!

JOHN

[fish7]

Steve,

I just thought of something.

Currrently EFS computation choices are ComputeOnClose or compute every tick.

Would it be possible for eSignal to create a ComputeOnPriceChange function to be called in PreMain? That would be a quick way to save CPU usage, rather than having the functions called at every tick. I'm sure there are many functions that only need to be called whenever the price changes, but not every tick.

JOHN

Hi John,

Your most welcome. Good to hear you liked the annotations.

If you look at the different methods in the EFS Knowledgebase, by specifying a bar index, a single value is returned, however if you do not specify a bar index, the methods return a series object. The series object will contain every single data point on the chart associated with the method. As you can imagine, returning every associated value on the chart would take more resources than returning a single value.

This may be a bit of an exageration to prove my point as there are checks that occur in the background to return a value if you expect a value... however, these additional steps add up. Testing I have done in previous versions has shown the adverse impact of not providing a bar index when executing these methods (aka functions). Add to that multiple executions of the same method, and the adverse impact on code efficiency is amplified even further (that's where the local variables can really help out with improving efficiency).

I was trying to show in the rewrite a technique that can be used to reduce efs cycling (in addition to the items already discussed). However, the efs was not itemized, I'll go into that thought process in the next post.

Originally posted by f1sh7
Steve,

Wow... thanks!... that was full of useful ideas.

I did a little test, and compared your faster version w/my original script, and it ran using 1/3rd the CPU time. That's amazing!

I had no idea that there would be a resource (and time?) difference between using close() versus close(0), especially since they return the same thing.

Now I will go and re-check all my EFS's and use more local variables. Did not realize that calling those functions more than once is slower than declaring a local variable.

Thanks a TON !!!

JOHN

John,

Your suggestion(s) is a potential improvement that makes sense to me, but I am not sure it is possible. I recommend you make that request to the eSignal Development Team, it may help to compensate for the impact of increasing trade volume.

As a follow-up to my last post, I rewrote the efs, simplifying it significantly. In tick playback, this version was two to three times faster than my previous markup.

I annotated my thoughts in the efs, hopefully they will make sense.

Originally posted by f1sh7
Steve,

I just thought of something.

Currrently EFS computation choices are ComputeOnClose or compute every tick.

Would it be possible for eSignal to create a ComputeOnPriceChange function to be called in PreMain? That would be a quick way to save CPU usage, rather than having the functions called at every tick. I'm sure there are many functions that only need to be called whenever the price changes, but not every tick.

JOHN

/********************************************************************
This EFS colors the volume bars red when they are above a
threshold level set by the user.
*********************************************************************/

var fpArray = new Array();

function preMain() {

setPriceStudy(false);
setStudyTitle("Volume Threshold Colorized Optimized");
setCursorLabelName("Vol",0);
setDefaultBarFgColor(Color.red,0);//~ changed default to red
setPlotType(PLOTTYPE_HISTOGRAM,0); 
setDefaultBarThickness(2,0);
setShowTitleParameters(true);
askForInput();
setShowTitleParameters(true); 
var x=0; 

fpArray[x] = new FunctionParameter("LargeVolThreshold", FunctionParameter.NUMBER);
with(fpArray[x++]){ // Vol above this is large, but not Mega
setLowerLimit(2); 
setDefault(2000);
}
}

function main(LargeVolThreshold) {

var myVolume=volume(0);//~ local variables are very very efficient and should be used instead of calling any method twice

if(myVolume<LargeVolThreshold){//~ only one conditional
//~ only execute the setBarFgColor(Color.black) function when Volume is below a level, i.e. expend
//~ resources when activity is lower, suspend function call (allow the default setting 
//~ to control bar color) when activity is higher, when you want reduced resource expenditure
setBarFgColor(Color.black);
}

return myVolume;
} 

[fish7]

Steve,

I was thinking more about resource/CPU usage, and was wondering if symbols in a Quote Sheet are updated every tick or not. I suspect they are.

IF SO .... then it seems a simple solution would be to add a feature to eSignal Quote sheets to allow the user to specify a 'refresh rate' for that specific quote sheet. This has two benefits: (1) On the user side it (1a) reduces bandwidth needs as less info is transmitted (1b) reduces CPU usage since every time a symbol is updated, any alerts the users have based on those symbols all need to be checked. (2) on eSignal side, it would lessen your demand on your servers.

I could envision a simple "right-click" to get an options window for any Quote Sheet. In that window, a refresh rate could be user-specified... such as once-per-second, once every 10sec, etc. Some quotes I would like to get every tick (e.g., $TRIN, Put/Call, etc.), but, my 100 or so stock symbols do not need to be refreshed that often... certainly not every tick.

Just a thought. Perhaps I am all wet. But figured I'd toss it out there.

JOHN