Optimizing EFS Strategies

[Doji3333]

I've had problems recently with some of my code and thought it was a problem with esignal - but it was not esignal's problem.

I have an efs script that was averaging 26.x + ms per tick in the EFS Performance Monitor. Most of my other efs scripts average 2.0 or less.

Well, this efs is very complex and builds lots of arrays of data that can sometimes contain old, unnecessary, data.

So, I wrote a routine to sort and remove the extra data and only keep the needed data in the arrays. This increased the performance of my efs from 26ms to under 5ms per tick. Now I'm very happy.

So, the lesson learned here... when in doubt, you need to optimize the performance of your EFS because it can have a dramatic effect.

This also tells me that storing and looping thru unnecessary data can be a very common cause of slow efs/esignal operation.

There is no simple trick to optimizing an efs. Some of the most common techniques are :

_ Limiting code operation to only when necessary (once per bar or only when a certain condition is met)
_ Limiting the amount of data that is contained/stored in arrays. This helps to speed up array operations (like loops/whiles)
_ Limiting the amount of DRAW functions that are issued in your code. I've learned to create "drawonce" and "removeonce" features so I don't have my efs code constantly REDRAWING things.
_ Limiting external calls (to dlls or other) to only when needed or using timers (like every 7 seconds) so these calls are not made every tick.
_ Setting STAGES of operation within your EFS to SKIP certain portions of code based on whatever stage/mode your efs is in. For example, if you're long, you don't need to look for new long trades.

I hope this helps some of you "power users" out as I've learned a valuable lesson today. What I thought was an esignal problem turned out to be a problem with my EFS scripts and holding TOO MUCH DATA.

The new script is currently averaging 4.033 ms per tick... A BIG CHANGE from 26.5ms per tick. Dang arrays!! :P

[ScottJ]

Thanks for posting your experiences and those solutions. As a power user, we really appreciate how you challenge the platform to it's fullest potential.

Thanks.

[Doji3333]

Thanks Scott. The code is only 5500 lines long and is one of the most complex efs scripts I've created. It's not big because I'm getting very good at developing code with arrays. But what it does is rather impressive. It's averaging about 7.5 pts a day in the S&P.

My other main robot is about 5000 lines of code and is averaging about 5 pts a day in the S&P (sometimes more).

As you can tell, I'm somewhat of a power user... LOL

Hope to retire soon while helping other esignal users. :P

[rdehavelyn]

IB Plugin "buyMarket()" parameters

Hi Brad,

My efs script has stopped executing trades throught the IB Plugin. The IB TWS recognizes the call, but does not execute. The trades get hung up in the TWS "API" folder, awaiting me to manually press "Transmit". It never used to do this, and IB says it's an eSignal issue.

I am using the latest Bridge and Charting app for eSignal, and my IB TWS is up to date as well.

My script simply uses the following code to go long IBM with 200 shares:
buyMarket(IBM, 200)

Any insight would be much appreciated.

Regards,
Ryan.

[Doji3333]

Ryan,

IB changed their API and "Order Flags" recently. They have a new 9.4+ API that they are distributing as a fix. They also have a new POSix API that is in BETA (that's what I'm using).

These new order flags are related to orders fired during the regular session vs. the overnight session. My MTS tool is still being tested as we've been working on a final solution. It may take a bit longer for me to get it fixed perfectly, but at least with the new and old MTS, I can still fire orders without any problems (for now).

IB does this every now and then (release a new API). And that means that esignal and many other (like myself) will have to go back into our code and "fix things" for this new API.

This is all part of the process or "maintaining support for brokers". Kind of a pain in the butt, but it's required.

Be patient as I'm sure esignal is dealing with this too right now. We all are.

[rdehavelyn]

eSignal "buyMarket()" / "sellMarket()" parameters for use with IB Plugin

Thanks for your reply. Can you share the required change to my order script that I will need to make for my trades to fire without being parked in the API folder of the IB TWS?

I spoke with IB, and they say that they now require the "transmit" flag to be populated (1=automatic, 0=confirm manually)

They say that eSignal needs to resolve this by re-writing their efs order paramaters, so the orders fire automatically. So the question is, has this been done?

Regards,
Ryan

[Doji3333]

You need to wait for esignal to fix it. This is not something I can fix for you. Sorry.

[eSignal_JasonK]

Hello Ryan,

Our QA group tested this scenario and everything appears to be working correctly. What version of the plug-in, eSignal, TWS and Java are you currently using?

[rdehavelyn]

Thx for the followup... I got on with Avery (one of your tech people) and he was very helpful.

Apparently the issue related to a prior install of eSignal that was not completely wiped from the registry. Required getting into the registry and killing all keys referencing prior IB Bridges. The other thing to note is when upgrading the main charting app it is a good idea to uninstal eSignal completely first. This means backing up any layouts, qote boxes and other custom preferences you may have to separate directory, then importing them into the new eSignal directory after the fresh install is complete.

I have the latest charting app (June 2), latest eSignal authored Bridge for IB works, the latest JRE (14) and everything is working great now.

Thanks.

[eSignal_JasonK]

Excellent. Thanks for the update.