Indicator processing speed

[gpfx19]

I'm noting that as I use the 2 following EFS files on a single chart, my entire computer speed radically slows down. My CPU usage goes to 100% & stays there for a while, not permitting me to enter or exit trades in a timely fashion. The attached Task Manager shot is typical, except CPU usage is chopped off at the top, like a square wave. This is NOT from process(es) that cannot be completed, as both EFS files, when run alone, work fine. They work fine here as well, but require too much CPU time.

I have a 1.6Ghz CPU & 500Mb of RAM. I have stripped the O/S of non-essential functions (email, MS Messenger, etc.). The PF Usage area is always a straight line, and never above 300Mb, so it appears to be a processing issue (please correct me if I am wrong).

In my testing, I have found that running anywhere from 4-15+ advanced charts concurrently doesn't seem to make much difference (regardless of the indicators on the charts), but using an indicator that has graphics or any backtesting (Strategy) functions, that I can tell, do appear to require a LOT more resources, hence this post. I would like to be able to determine which functions require the most resources.

Is there a means to note precisely the CPU requirements for an EFS file? (Using just the task manager isn't good enough) If said file is using the page file, & if so, how much of the page file is being used? How much RAM is required to run an EFS file? And so on. You know, like main frames used to give you each time you ran a program. ;-)

TIA,
Greg

[gpfx19]

SuperBLM file

[gpfx19]

Brad's file.

[gpfx19]

Task manager shot.

[gpfx19]

Task manager doesn't appear (why not?), and none of them would attach to the original post either. 'Cuse the wasted bandwidth.

Anyway, thanks for any guidance you can give.
- Greg

[gspiker]

Hi gpfx19,

The Super indicator should be pretty light weight aside from all the shapes its draws. If you have a large number of bars up, or are running this on a large number of charts I guess it could cause a few problems...but running on my 90 day 5Minute ES chart I didn't really have a problem. Too many lines, shapes or images (useing the drawLinexxx(), drawShapexxx() or drawImagexxx() ) can cause a system to slow down due to the overhead of eSignal processing all the draw data.

The ment-ma EFS also uses drawShapeXXX() functions. Again unless you are running it on a large number of bars or charts I wouldn't expect any major problem from this. However it also uses the getValue("time") function...which is VERY resource intensive and no longer the recommended method for getting time data. If you are running this EFS on just a few charts, I would expect a big slow down. Again for informational purposes running it on just one 90 day 5 minute ES chart didn't cause any major slowdowns on my computer.

If you want to speed things up a bit, you could change the getValue("time") function to use rawtime instead...or better yet use the new eSignal functions getHour() and getMinute(). These use a less resources, but are still not exacly "light weight".

Hope this helps...

Garth

[eSignal_JasonK]

Hello Greg,

You may also want to review the following KnowledgeBase article, eSignal - Freezing and Slow Performance Issues for some more insights into your performance issues. You can also view the performance of each individual EFS through the EFS Performance Monitor under the Tools-->EFS menu to help identify the least efficient formulas and functions running in your layout.

[its1111]

hi Jason
Great answer and thread, I did look under tool bar and the preformance button, but it didn't do anything. I am sure I am doing something wrong, any help would be appreciated on how to test each EFS
Thanks

Mark

Mark,

Check out this thread http://forum.esignalcentral.com/show...itor#post28864 as it contains a real good discussion on how to access and use the performance monitor. Ensure when you select the window that the performance monitor has the enable button selected. Additionally, if the performance monitor is not appearing at all (potentially offscreen), Alex has posted a solution to that issue as well. Hopefully this helps.

[bpschoch]

I recently rewrote the CCI trading panel for Woodies CCI Club and was able to write it to consume the minimum amount of processor overhead even though it's over 3000 lines of efs code. These are some of things I learned.

the 'main' function is called on every tic but most studies don't need to do something on every tic. In fact there are 4 conditions that a study might want to do something in a chart and the tic condition is probably the least important. I divide these 4 conditions as follows:

1) one time when chart is loaded
2) once per bar
3) when there is a price change
4) when a tic occurs

Most studies only are really interested in the first 3 conditions. However if you are plotting or are dynamically changing the color of plotted lines, you do need to be aware of the tic condition.

This is how I structure my efs's:

var OneTimeSwitch = true;
var prevClose = -1;
var ReturnArray = new Array(4); // assume 4 return parameters
var ColorLine1;

main(parms......) {
if (OneTimeSwitch) { // this is the once per study (case 1)
OneTimeSetup(parms......);
OneTimeSwitch = false;
}
if (getBarState() == BARSTATE_NEWBAR) {
// case 2 once per bar
// I do all my once per bar processing here and any close out of last bar
}
var curPrice = close(); //get current price
if (curPrice != prevClose) { // case 3 price changed
// do any processing that you need to do on a price change
// this includes getting any values from builtin studies
// if you need to change the color of a line based on price,
// set it here
// also set any return values
ColorLine1 = color.red;
ReturnArray[0] = value of line 1
ReturnArray[1] = value of line 2
.....etc
prevClose = curPrice;
}
// the per tic case 4
setBarFgColor(ColorLine1,0); // unfortunate this has to be done here
return ReturnArray;
}

As you can see, this separates the study into the 4 conditions. It's important to move everything up to the highest level possible and to minimize anything done at the tic level. For example, many studies may need to loop through previous bars and this can usually be done at the once per bar level. (When you are within a bar, the previous bars don't change). I hope this gives people some ideas on how to improve the performance of their efs studies.
Bernie Schoch

[gspiker]

Most studies only are really interested in the first 3 conditions. However if you are plotting or are dynamically changing the color of plotted lines, you do need to be aware of the tic condition.

I would argue that any futures trading system, run on any chart of significant interval, needs to do a stop check on each tick. Anyone who has traded through a crash knows that you can blow though a reasonable stop intrabar.

There are also a number of other reasons to do so...including intrabar predictive filters or running systems that can signal without an end of bar condition.

Garth

[bpschoch]

my point is that those trading decisions can still be made at the price change level and not the tic level. By my definition, the tic level is when a tic occurs but the price is the same as the previous tic; most of the time, studies don't need to do any processing in that condition (there hasn't been a price change). In my experience, a significant amount of calls to main are made with the same price as the previous tic and these code filters out that condition.

Bernie

[gpfx19]

Jason,
Thank you for the information on the Performance Monitor. It is the start of what I requested (see more below).

Benie,
Excellent! Thank you much for having shared your experiences of stepping back & taking a look at the bigger picture. You've pushed me forward at least 2 big steps. I'm looking forward to the structural changes in my code you've suggested, though not the time involved. ;-)

Garth,
I immediately implemented the changes you suggested (before Jason's Performance Monitor information). Changing to the getHour() & getMinute() gave a slight decrease in resources requried (so many thanks for something I would not have known otherwise), but commenting out the drawShapeXXX() functions gave a 50%+ reduction in resources required. I've since verified this with Performance Monitor.

Jason - is there further information on how Performance Monitor works? The reason I ask is because in verifying my results from Garth's suggestions, I note that 'Total Calls' appears to have the only correlation with Task Manager, and so my computer resources, versus either of the 2 time columns (which would make sense if making more calls places a greater demand on the system, even though total & average time to process said calls might be less - please correct me if I am wrong).

Further, Total Time & Avg. Time don't appear to engage until I edit the study, in any fashion, at least not within 10 minutes of doing a reset. Avg. Time is where the time doesn't appear to have any correlation to system speed or resources either. On my machine, ment-ma has an average time ~1/2 of SuperBLM, while SuperBLM has over 5 times as many calls as ment-ma. Any one else's thoughts, knowledge, &/or observations would be greatly appreciated.

Steve,
In the Performance Monitor, I also note a Global Stat of "addShape" that I would assume is your "Edit" button that I use in my of scripts. Thank you for the code on how to do this, BTW! I've been using it since I found that thread.

Thanks all,
Greg

[eSignal_JasonK]

Hello Greg,

The Performance Monitor is pretty basic as far as the information displayed. You're not going to be able to directly relate the performance of a specific EFS to the information in the task manager because the task manager is only application specific or process specific. An individual EFS will not appear as a separate process. In general, less calls and less time per call equates to greater efficiency in terms of CPU usage. When you are developing formulas and you want to measure the affect on Average Time per call after making some changes to your formula, save the formula, click the Reset button in the Performance Monitor and then reload your formula. That will refresh the stats to reflect your recent code modifications. If you only want to measure the affect of those changes in streaming mode then save your formula, reload it and then click Reset in the Performance Monitor.

One note regarding the use of debugPrintln(). If you have an application that is outputting to the formula output window, it can have a significant negative impact on efs performance. Please check this link http://forum.esignal.com/showthread....ndow#post48331 for a discussion regarding the use of debugPrintln() in addition to other discussions related to efs performance.

With regards to evaluating prices and stop levels every tick, remember that main() still executes every tick. Putting some steps in main that execute every tick that are simple in nature has very little impact on execution speed. Putting steps that refer to the date object, output to the formula output window using debugPrintln or open and close files (my trace utility for example) are performance killers. You just have to be careful. Personally, I check to see if my stops and profit targets are hit every tick (as Garth recommends) and it has very little impact on execution speed using the performance monitor (evaluated similarly to how Jason recommends below) .