why does this happen?

[clearpicks]

I noticed sometimes in the performance monitor window the total running time jumps mysterious. So I striped my trading script down to a very simple one. Because I suspect it is caused by the sym() and inv() in EFS2, so the testing script only display the volume of another symbol. ( I tested it on "NG K8=1,1" chart to show the volume of "UNG". I don't know why in eSignal status bar it should "Sym 3". Anyway, the screenshot displays all windows on my screen. In the UNG,1 chart, the esignal basic study "Volume" was applied, while on NG K8=1,1 chart, my testing script was applied.
In the formula output screen, when UNG volume is accessed by setting the value of the variable "instrument" to "UNG", it can be seen sometimes the testing script printed out non zero dms, which really puzzled me. While when setting "instrument" to "NG K8=1" so that only volume(-1) was called, then nothing was printed out (as long as my test was running). Also it seems the non-zero dms values were printed out when the new incomming ticks were the first tick in a minute (not 100% sure about this), so I suppect that the implementation of sym() and inv() in EFS2 engine somehow comsumed too much CPU power while trying to allign the series of two symbols ( basically NG has data for the new minute while UNG does not yet or vice versa.

Another thing I can not explain is it seems when more charts running at the same time, for example, with CL K8,1 chart to show USO, ZG M8,1 to show GLD,1 volume, etc., the value of dms printed out may decrease. I have not find out the pattern yet. Generally the value of dms printed out may vary from about 100ms to 15ms (the minimum I observed).

Please confirm what I observed and give some explanation. If this was really caused by the EFS2 implemeation bug of sym() and inv(), it may help to lower the lagging and slowing of eSignal in some scenarios.


- Clearpicks

[clearpicks]

screenshot

I have to cut it into two files.

[clearpicks]

screenshot part 2

[clearpicks]

I modified the testing script a little bit by delaring closeArray, volumeArray, etc. as global variables (when I was preparing the testing script this morning I deleted those lines from my trading script accidently). However according to Jason's post, EFS2 should be able to maintain those serieses created by calling sym() and inv(). The result is the same as I expected, there is no difference after adding those lines.

Anyway, the reason I think EFS2 engine is questionable is that now I can almost 100% sure when I would observe a total time leaping in EFS performance monitor (the printed value of dms is equal to the total time increment of the tesing script, and it seems such "leaping" only happens at the first tick of each minute.


Attached is another screenshot. dm is the difference between minute of last bar of NG K8=1 and UNG. Because right now the UNG is not traded and each new bar of NG would change the value of dm at least by 1 (because last 1min bar of UNG is at 16:10 and mUNG is freezed at 10).

So it seems EFS2 comsumes way too much CPU power to align the serieses of two symbols at the beginning of each new bar. When there are many charts with short intraday time interval like 1min or 2min and use sym() and inv() to access other time frame or symbols, such short term CPU usage spike may potentially cause troubles such at data loss, lagging and esignal freezing.

In the attached screenshot, there are two instances when dm changes but dms is still equal to zero. These happen at 16:14 and 16:17, there are two bogus bars on UNG charts, and there is no trades on NG K8 charts. so the value of dm changed from 2 to -2 and from 3 to 0 but cause no non-zero dms printed out. In all other cases, new bars of NG were created and there were no matching UNG data for those new bars, and EFS2 engine spent (wasted) about 100ms each time here.

This test was done using eSignal 10.0.1086.932. I am not sure whether anything has been changed in 10.1 beta.

- Clearpicks

[clearpicks]

The modified testing script.

[clearpicks]

Running it during the night when UNG is not traded, it is very easy to identify when non-zero dms is printed out. I modified the script to print out the HH:MM info of the last bar of NG and UNG.


- Clearpicks

[clearpicks]

The script was not attached.

Originally posted by clearpicks
Running it during the night when UNG is not traded, it is very easy to identify when non-zero dms is printed out. I modified the script to print out the HH:MM info of the last bar of NG and UNG.


- Clearpicks

[clearpicks]

The script was modified so that only volume(sym(volSymbol)) was called once.

I ran the testing script on YM M8 because there was almost no trade on NG K8 at that time. I changed the time template from 2Days 24 hour to 10Days 24 hours. The value of dms changed from about 150ms to 950ms which is roughly proportional to the number of bars on the chart.

So it is very likely this is caused by bad data structure or algorithm in EFS2 engine.

- Clearpick

[clearpicks]

I modified the testing script so that volume(sym(volSymbol)) was called at each tick. There is no noticable difference regarding to the dms value printed out compared to calling it only once and using the variable "volumeArray" to keep a reference to the created series, which means as JasonK said EFS2 is able to "reuse" the existing volume series of "UNG,1" instead of rebuilding the series when each time volume(sym(volSymbol)) is called. However it is obvious something is wrong in sym(). I will test inv() and report later.

[clearpicks]

It seems inv(0 has the same problem.

[clearpicks]

The script to test inv().

[clearpicks]

It seems whenever there is a new bar being created, either of the primary symbol or the secondary symbol, an non-zero dms is printed out (which means EFS2 is wasting some thime doing something). From attached screenshot, it can be seem that at local time 13:50:01, when a YM M8 new bar was created for 13:50, the time stamp pf the last bar of UNG was still 13:49, EFS2 ran for 125ms to process that tick. At 13:50:18, a trade of UNG came in and a new bar of UNG was created. This time EFS2 spent another 125ms doing something mysteriously.

eSingal, please test it and give me a valid explanation.


- Clearpicks

[clearpicks]

The testing script.

Originally posted by clearpicks
It seems whenever there is a new bar being created, either of the primary symbol or the secondary symbol, an non-zero dms is printed out (which means EFS2 is wasting some thime doing something). From attached screenshot, it can be seem that at local time 13:50:01, when a YM M8 new bar was created for 13:50, the time stamp pf the last bar of UNG was still 13:49, EFS2 ran for 125ms to process that tick. At 13:50:18, a trade of UNG came in and a new bar of UNG was created. This time EFS2 spent another 125ms doing something mysteriously.

eSingal, please test it and give me a valid explanation.


- Clearpicks

[clearpicks]

I just tested it in eSignal 10.1 beta and it has the same problem.
I uninstalled esignal 10.1 and installed eSignal 8.0 build 782, which is a pretty old one. As I expected, the dms value printed out was always zero and eSignal loaded much faster. That is, EFS2 engine in eSignal 8 spends little time processing sym(0 and inv() while in eSignal 10.x related code was modified so that a BIG bug was introduced. Maybe this is the reason why many people complaint about the slowi loading time of eSignal 10.x. I tested sym() and inv() long time ago so that the running time of my trading scripts were decreased to very small value. After upgrading to eSignal 10, I observed various problems and slowing of my scripts and loading time. Because my trading scripts were kept evolving, I thought it might be caused by my scripts and new features of eSignal 10. However it seems something has been changed in sym() and inv() implementation since eSignal10 without extensive testing and public announcement. eSignal, next time when you change any code, please make sure it does not make existing things worse.

I spent a lot of time trying to pin point each esignal bug I encountered so as to help myself and all other fellow users. Anyway, a better and faster eSignal is all what we want. I don't understand why there is no reply to my posts in this thread and my attached EFS files were downloaded onoly 1 to 3 times respectively, and most of these downloads were done in midnight.

- Clearpicks

[eSignal_JasonK]

Hello Clearpicks,

I will be looking into all of the information you have posted. At this time I do not have specific answers for you.